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Message from 


the Director 


Iam pleased to present the annual 
report on provincial achievement 
testing for the June 1992 
administration. Results were 
mixed. In Grade 3 Social Studies, 
close to 84 per cent of the students 
achieved the acceptable standard, 
and a significant percentage 
achieved the standard of excellence. 
A smaller percentage of students 
achieved the standards in Grade 6 
Language Arts and Grade 9 
Mathematics. 


Results in Grade 9 Mathematics 
were particularly disappointing, 
especially in problem solving, 
where approximately 64 per cent 
of the students achieved the 
acceptable standard. The Grade 9 
Math results are consistent with 
the poor Grade 6 Math results in 
1991 and diploma exam results in 
Math 30. All these results suggest 
that students are not developing the 
sound conceptual basis they need 
for success in mathematics. 


Achievement-over-time results are 
again included in this annual 
report. This year, we compared 
student achievement in 1992 with 
that in 1988 and 1984. The data 
indicate improvements in Grade 3 
Social Studies and Grade 9 
Mathematics performance 
compared with student 
performance in 1984. 


This report includes results of 


French immersion students and of 


Francophone students on the 
provincial achievement test for 
Frangais 6° année. Close to the 
expected number of students 
achieved the standards. 


Also in this year’s report is 
information from performance- 


based assessment carried out with 
a sample of the students in each of 


the grades. Section 2 provides 
results for participation skills 


(Grade 3 Social Studies), listening 


and viewing skills (Grade 6 
English Language Arts), and 
problem-solving skills (Grade 9 
Mathematics). Included in this 
section as well is information 
obtained from questionnaires 
administered to a sample of 
students and teachers about the 
contexts for students’ learning. 


I want to express our appreciation 
to teachers, principals, and 
superintendents who helped us 
carry out these provincial 
assessments. I trust that this report 
will be interesting and useful to all 
of you. I hope it will assist you in 
reflecting on your instructional 
programs and in encouraging all 
students to meet high standards of 
achievement. 


th, 


Frank G. Horvath, Director 
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Section 1 


Summary of 
Achievement Test Results 


Parents, educators and the general 
public need to know how well 
Alberta students are achieving in 
relation to provincial standards. 


The 1992 achievement test results 
help demonstrate what is possible 
for grades 3, 6, and 9 students to 
know and do in Social Studies, 
Language Arts, and Mathematics 
respectively. 


Knowledge of what is possible 
produces new enthusiasm, 
raises sights, establishes new 
challenges and ultimately can 
improve personal and societal 
performance.* 


This first section of the report 
describes certain broad 
characteristics of the student 
population who wrote the 
achievement tests and provides 
answers to the following questions: 


¢ How many students wrote 
the achievement test for 
their grade and how many 
were absent and exempt? 


What percentage of the 
students attained the 
acceptable standard or 
higher according to criteria 
set by Alberta Education? 


¢ What percentage of the 
students attained the 
standard of excellence or 
higher according to criteria 
set by Alberta Education? 


Results are reported in terms 

of three related but different 
standards: curriculum standards, 
achievement standards, and 
assessment standards. 


Curriculum standards are the 
expected student learnings, 
sequenced into grade levels, that 
are stated for each curriculum. 
They include specific statements of 
knowledge, skills, and attitudes 
against which student performance 
is to be judged. 


Achievement standards, usually 
expressed as percentages, state 
how many students at a given age 
or grade in school are expected to 
achieve or exceed the acceptable or 
excellent levels. It is important to 
point out that this judgment is not 
a prediction of what percentages of 
students will actually achieve or 
exceed acceptable or excellent 
levels of performance. 


These achievement standards 
apply to school, jurisdiction, and 
provincial performance. 


Assessment standards are the 
scores to be achieved by a student 
on a specific test or part of a test 
before the performance of that 
student is judged to be “acceptable” 
or “excellent” relative to the 
curriculum standards. Alberta 
Education reports the results for 
the achievement tests in relation to 
an acceptable standard and a 
standard of excellence. 


Assessment and achievement 
standards for the 1992 achievement 
tests were determined and 
subsequently recommended to the 
Director of Student Evaluation by a 
curriculum and test development 
committee, a test review 
committee, a public advisory 
committee, and experienced subject 
classroom teachers who applied 
standard-setting procedures to the 
tasks under the guidance of the 
Analytic Services Unit of the 
Student Evaluation Branch. An 
outline of the processes followed is 
provided in Appendix A. 


The public review of standards is 
described in Appendix B. 


Guidelines for interpreting the 
1992 results are given in 

Appendix C, and Appendix D 
provides answers to a number of 
questions frequently posed by 
parents. The process of developing 
the achievement tests is outlined in 
Appendix E. 


Student Populations 


In June 1992, principals reported 
a total population of 113 919 
students in the grades tested. 
There were 41 696 students in 
Grade 3, 37 837 students in 
Grade 6, and 34 386 students in 
Grade 9. Of the total population, 
97 315 wrote achievement tests. 
The remaining 16 604 students 
were exempted from writing or 
were absent the day the tests were 
administered, as shown in 


*Learning Mathematics/Learning Science International Assessment of Educational Progress, Educational Testing Service, 


February 1992. 


Figure 1-1 


Students Writing Achievement Tests 


dune 1992 


100 


Social Studies 


English Language Arts Mathematics 
(Gr. 3) (Gr. 6) (Gr. 9) 
N=41 696 N=37 837 N=34 386 
G 
Students writing the test* LL Students absent or exempt 


*The number of students writing the tests includes students in the regular English program, 
Francophone and French Immersion students who wrote the achievement test 
in either English or French, and Grade 3 Social Studies students in combined (split) 


classes who wrote only a partial test. 


Provincial results are based on test scores achieved by students who were in the regular 
English program, which is defined in the footnote for tables 2-2, 3-3, and 4-2. Results for 
Francophone students, French Immersion students, and for students in combined (split) 

classes were reported directly to participating schools and jurisdictions and are therefore 


not calculated in provincial results. 


Figure 1-1. The tests were 
administered in three special forms 
in addition to the regular form. 
The information collected from the 
special forms was used to 
investigate student achievement 
over time. These results are 
reported in Section 8 of this report. 


Observations on Student 
Participation in 1992 
Achievement Tests 


The student participation rate was 
highest in Grade 6 (89.1%) and 
lowest in Grade 3 (82.1%) when 
calculated on the numbers 
gathered from the annual 
Principal’s Reports submitted 
after each achievement test 
administration. However, the 
lower participation rate in Grade 3 
Social Studies may be because 
there was no French translation 


of the test for those students 
instructed in French. In terms of 
absolute numbers, more Grade 3 
students (34 229) wrote their test 
in Social Studies than did Grade 6 
students in Language Arts (33 717) 
and Grade 9 students in 
Mathematics (29 369). 


When participation rates are 
calculated from the September 30, 
1991, enrolment data, the rates 
change slightly. In Grade 3 Social 
Studies, the participation rate 
marginally decreased to 82.0%; in 
Grade 6, it increased to 89.8%; and 
in Grade 9, it fell to 82.9%. 


The difference in participation 
rates, based on these two methods 
of calculation, was much smaller 
than the differences found in the 
1991 data. 


The achievement test 
administrative accounting was 
enhanced for the 1992 
administration. For the first time, 
all schools were required to report 
student counts for each grade 
tested whether they were 
presenting students for testing 

or not. 


Reports were required for: 


* the total number of students 
enrolled in the grade on test day, 

¢ the number of students who 
wrote the test, 

¢ the number of students who were 
absent on test day, 

° the number of students who were 
exempted from writing the test. 


Compared to previous years, this 
method of data collection used on 
test day in June 1992 resulted in 
the reporting of: 


¢ a higher total number of 
students, 

¢ a higher number of students 
who were absent on test day, 
and 

¢ a higher number of students who 
were exempted from writing the 
test. 


Details of student participation 

at each grade level, showing the 
number of students who wrote the 
tests, who were absent, or who 
were exempted, are presented in 
sections 2, 3, and 4. 


Results in Relation 

to Standards for 
Students Who Wrote the 
Achievement Tests 


For the 1992 results, the focus of 
reporting student achievement is 
on the major components of the 
test as well as on the total test. 
Results are reported in relation 
to the standards established for 
individual major components as 
well as for the total test. 


From discussions with educators, 
test development specialists, and 
curriculum specialists, and based 
on our experience with measuring 
student achievement according to 
the expectations of the Program of 
Studies, we expect 85% of students 
to achieve the acceptable standard 
or higher and 15% of students to 
achieve the standard of excellence 
or higher on each major component 
and on the total test. 


Standards reflect expectations 
for the regular English language 
program students and not for the 
total population—which, for 
example, would include special 
needs students. 


Figures 1-2 and 1-3 present 

the percentage of students who 
achieved the acceptable standard 
or higher and the standard of 
excellence or higher based on their 
total test scores. 


Figure 1-2 
Percentage of Students Achieving Acceptable Standard or 
Higher* on the Total Test 
June 1992 
100 
80 
% 
40 | 
20 + 
0 
Social Studies English Language Arts Mathematics 
(Gr. 3) (Gr. 6) (Gr. 9) 
N= 31 654 N=33 717 N= 27 888 


*Includes students achieving the standard of excellence or higher 
**85% of students were expected to achieve the acceptable standard or higher 
on the total test. 


Figure 1-3 

Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test 

June 1992 


Social Studies English Language Arts Mathematics 


(Gr. 3) (Gr. 6) (Gr. 9) 
N= 31 654 N= 33717 N= 27 888 


*15% of students were expected to achieve the standard of excellence or higher 
on the total test 


General Observations 
on Results 


Of the three grades and subjects 
tested in 1992, the results in 
Grade 3 Social Studies were closest 
both to reaching provincial 
achievement standards and to 
being judged satisfactory. In 
Grade 3 Social Studies, results 
based on the total test scores 
revealed that the percentage of 
students achieving the acceptable 
standard or higher was slightly 
lower than expected, and the 
percentage achieving the standard 
of excellence or higher was 
marginally higher than expected. 


Results for Grade 6 English 
Language Arts were disappointing. 
The percentages of students 
achieving standards were much 
lower than expected for both 
levels. 


Results for Grade 9 Mathematics 
were the most disappointing. The 
percentages of students achieving 
standards for either level were 
much lower than expected and 
were the lowest of the annual 
program results. 


Section 2 
| 


Grade 3 
Social Studies 


Introduction 


In 1992 a number of assessments 
were carried out in Grade 3 Social 
Studies. The achievement test was 
administered to students province- 
wide. Participation skills were 
assessed with a sample of Grade 3 
Social Studies students. As well, a 
sample of Grade 3 Social Studies 
teachers and a sample of Grade 3 
Social Studies students 
participated in a pilot study to 
examine the relationship among 
various contexts for learning and 
their effect on achievement. The 
results from all these assessments 
follow. 


Achievement Test 
General Description 


The Grade 3 Social Studies 
Achievement Test consisted of 

50 multiple-choice questions. The 
time allotted for writing the test 
was 20 minutes for each of three 
parts. Statistics for the total test 
and for the components are based 
on the results achieved by 

31 654 students: 29 043 wrote 
the regular form and 2 611 
participated in the achievement- 
over-time study, as shown in 
Table 2-2. This section of the 
report provides answers to the 
following questions. 


* How many Grade 3 students 
wrote each form of the test or 
were absent and exempt? 

¢ What percentage of Grade 3 
students attained the acceptable 
standard or higher in Social 


Studies according to provincial 
criteria? 

What percentage of Grade 3 
students attained the standard 
of excellence or higher in Social 
Studies according to provincial 
criteria? 

What did Grade 3 students know 
and what could they do in Social 
Studies? 

What parts of the Social Studies 
curriculum caused Grade 3 
students difficulty? 


Summary of Results 


Results in Relation to 
Standards 

Results show that 83.5% of 
students who wrote the test 
achieved the acceptable standard 
or higher and 15.9% achieved the 
standard of excellence or higher on 
the total test. These results were 
slightly lower than expected for the 
acceptable standard but marginally 
higher than expected for the 
standard of excellence. 


Average Score 

The average total test score was 
77.4%, with a standard deviation 
of 16.5. The average total test raw 
score was 38.7 marks out of a 
possible 50, with a standard 
deviation of 8.2. 


Content of the Test 


The Grade 3 Social Studies 
Achievement Test was designed to 
reflect the Grade 3 Social Studies 
Curriculum (revised 1990). The 
scope of the test was limited to 
knowledge and skill objectives that 
could be effectively measured on a 
multiple-choice test. As a result, 
all questions were drawn from the 
content of the three topics 
prescribed for Grade 3: 


Topic A: My Community in the 
Past, Present, and Future 

Topic B: Communities Need Each 
Other 

Topic C: Special Communities 


The knowledge objectives 
component consisted of questions 
from the three topic areas in Grade 
3 Social Studies. The skill 
objectives integrated knowledge 
with process skills and 
communication skills. 


Questions measured student 
achievement in two cognitive 
levels: 


¢ Knowledge—recognize or recall 
ideas, terminology, facts, 
principles, generalizations, 
and concepts 


* Process Skills—locating/ 
organizing/interpreting 
information; geography and 
mapping; analyzing/synthesizing/ 
evaluating 


Test Blueprint 


The test blueprint shows the 
distribution of questions according 
to the curricular content area 
(topic) being assessed and 
according to the knowledge and 
process skills required to answer 
the question. 


Table 2-1 

Grade 3 Social Studies 
Achievement Test Blueprint 
June 1992 


Topic/Concept Reporting Category 


Topic A Topic B Topic C 
My community Communities Special 
in the past, need each other communities Proportion of 
Objectives Reporting Category present, and Total Score 
future 
Knowledge Objectives 
Understands generalizations, concepts, 1,2,4,5,6,10 19,21 ,22,23,31 34 35 ,36,37,39, 36% 
related facts and content 43,49 
Process Skill Objectives 
Locating/organizing/interpreting 3,7,11,12,15 20,24 ,26 40,41,45,48 24% 
information 64% 
Geography and mapping 16,17 25,28,29,30 46,47 16% 
Analyzing/synthesizing/evaluating 8,9,13,14 18,27 ,32,33 38,42,44,50 24% 
Proportion of Total Score 34% 34% 32% 100% 


Student Participation Results for students in French 
In June 1992, principals reported Immersion va ae i 
a total population of 41 696 programs or in combined (split) 


studente im Grade 3. Table 2:2 classes who wrote a partial test 
: only are reported separately to 


participating schools and 
jursidictions. 


presents the number and 
percentage distribution of students 
who wrote the Grade 3 Social 
Studies Achievement Test, who 
were absent, and who were 
exempted. 


Table 2-2 
Grade 3 Social Studies 
Student Participation : 
June 1992 
Number Percentage 
Category of Students of Students 


Other Students Who Wrote:*** 2 575 6.2 
Francophone—Taught in French, Wrote in English 24 0.1 
French Immersion—Taught in French, Wrote in English 634 1.5 
Combined (Split) Classes—Wrote Partial Test only 1917 4.6 
Students Absent 1327 3.2 
Students Exempted from Writing 6 140 14.7 


Categories of Exemption: 


1. Special Needs Students 1 224 2.9 
2. Subject Was Not Taught This Term 947 2.3 
3. English as a Second Language Students 501 1.2 
4. Language of Instruction Was Not English 1 769 4.2 
5. Other (as approved by the Superintendent) 1 699 4.4 
Total Principals’ Reported Population 
Test Day, June 1992: 41 696 100.0 
Grade 3 Enrolment: September 30, 1991 41 763 


* Provincial results are based on test scores achieved by students who were in the “regular” Social Studies program. Such students are 
defined as those who were instructed in English and whe wrote the English form of the achievement test, and those students who were 
instructed in a language other than English or French and who wrote the English form of the test. They are indicated by the shaded area 
of the table. 


** Regular Form refers to the unmodified 1992 test. The modified tests are described in Section 8 of this report. 


*** Results for Francophone or French Immersion students or for students in combined (split) classes who wrote partial tests only are reported 
directly to participating schools and jurisdictions and are not calculated in the provincial results. 


Results in Relation 
to Standards 


Figures 2-1 and 2-2 and Table 2-3 
show the percentage of students 
achieving the acceptable standard 
or higher and the standard of 
excellence or higher on the total 


test and on components of the test. 


The levels of performance were 
higher than expected for the 
standard of excellence on all 
reporting categories and lower 
than expected for the acceptable 
standard on the total test and on 
the knowledge component. 


Ess SSSarass tas Sane aaa ea 


Figure 2-1 

Grade 3 Social Studies 

Percentage of Students Achieving Acceptable Standard or 
Higher on the Total Test and on Components of the Test 
June 1992 


100 
80 


Process Skills 


Total Test Knowledge 


Actual Results** 


(“7A Achievement Standara* 
Y See Appendix A 


* the percentage of students in the province expected to meet the acceptable standard or 
higher 
** the percentage of students in the province who met the acceptable standard or higher 
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Figure 2-2 

Grade 3 Social Studies 

Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test and on Components of the Test 
June 1992 


Process Skills 


Total Test Knowledge 


WY Achievement Standard* Actual Results** 
Y, 


See Appendix A 


* the percentage of students in the province expected to meet the standard of excellence or 
higher 


** the percentage of students in the province who met the standard of excellence or higher 


Table 2-3 


Grade 3 Social Studies 
Students Achieving Standards on the Total Test and on Components of the Test 
June 1992 
Provincial Students Achieving Assessment 
Maximum | Assessmen Standard or Higher 
Possible Standard* Expected Actual 
Reporting Category Score (Raw Score) Number Number 
Standard of Excellence or Higher 
Total Test 50 
Facts, Concepts, Generalizations Only 18 
Process Skills Only 32 
Acceptable Standard or Higher 
Total Test 50 
Facts, Concepts, Generalizations Only 18 
Process Skills Only 32 
Below Acceptable Standard N/A 
on Both Components 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the lowest score a 
student must achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. 


See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provincial Assessment 


Standard. See Appendix A. 


It should be noted that the actual 
percentages of students achieving 
or exceeding standards on the total 
test for Grade 3 Social Studies are 
based on the 31 654 students in the 
regular programs who wrote the 
test. 


If, however, the percentages are 
based on the total population 
reported by principals (41 696), 
less the French Immersion, 
Francophone, and combined (split) 
class students who wrote the test 
(2 575), the percentages achieving 
standards on the total test would 
be: 

12.9% achieving the standard of 

excellence or higher 

67.6% achieving the acceptable 

standard or higher. 


If the percentages are based on the 
September 30, 1991, Grade 3 
enrolment (41 763), less the French 
Immersion, Francophone, and 
combined (split) class students who 
wrote the test (2 575), the 
percentages achieving standards 
on the total test would be: 

12.9% achieving the standard of 

excellence or higher 

67.4% achieving the acceptable 

standard or higher. 


It is emphasized that the above 
percentages, based on the total 
population and enrolment figures, 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for would have achieved standards. 
The absence of information on 
these students is nonetheless 
problematic. 


The number of students achieving standards were those for which 5% or smaller probability that a 


the acceptable standard or higher the difference between the actual difference of that size could occur 
and the standard of excellence or number of students and the by chance. The results are 

higher for each jurisdiction was expected number of students at reported in Table 2-4. The 
analyzed to determine whether or above standards was not percentage distributions in 
jurisdictions were below, meeting, statistically significant. A 95% the table are based on 198 

or exceeding provincial achievement confidence interval was used; this jurisdictions (including private 
standards. Jurisdictions classified criterion means that differences schools). 

as meeting provincial achievement are only reported when there is a 


Soe 


Table 2-4 

Grade 3 Social Studies 

Percentage Distribution of Jurisdictions* Meeting Provincial Achievement Standards on the 
Total Test and on Components of the Test 


June 1992 
Percentage Distribution of Jurisdictions 
Not Meeting Provincial Meeting Provincial Exceeding Provincial 
Achievement Achievement Achievement 
Component Standard (Per Cent) Standard (Per Cent) Standard (Per Cent) 

Standard of Excellence or Higher 

Total Test 9.6 75.8 14.6 

Facts, Concepts, Generalizations Only 45 58.6 36.9 
Process Skills Only 3.6 62.1 34.3 
Acceptable Standard or Higher 

Total Test 26.3 63.1 10.6 

Facts, Concepts, Generalizations Only 33.8 57.6 8.6 
Process Skills Only 19.2 65.1 15.7 


*Jurisdictions with fewer than five students are excluded because the statistical significance of the difference between the number 
actually meeting or exceeding the standard and the number expected to meet or exceed the standard when calculated and reported is not 
educationally meaningful. 


Results for Individual The results shown in these tables Statistics presented in these tables 

Mi ultiple-Choice and can best be used in conjunction are based on results achieved by 

Mapping Questions with the tables in the jurisdiction 29 941 students (those who wrote 
and school reports. Variations in the regular form and those who 

Tables 2-5A to 2-5C show the patterns of students’ responses to wrote Form F). 

percentage of students who chose questions can help to indicate 

each alternative (A, B, C, and D) strengths and weaknesses in local The sum of the percentages for 

for each multiple-choice question. educational programs. each question may be less than 


100% because the No Response 
category is not reported in these 
tables. 


Table 2-6 shows the percentage of 
students who gave answers to each 
mapping question. The correct 
response (key) for each question 

is identified; the component, the 
category, and the curriculum 
standard that each question 
measures are also indicated. 
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Table 2-5A 

Grade 3 Social Studies 
Results for Individual Multiple-Choice Questions 
Topic A: My Community in the Past, Present, and Future 


June 1992 
Distribution of Responses (%) 
Item Component Category Curriculum Standard 
A B c D 
1 86.1" 5.0 3.3 4.0 Knowledge Cc defines what a community is 
2 71,4" 18.9 6.2 2.7 Knowledge Cc identifies an essential characteristic 
of a community 
4 87.7* 2.4 5.8 3.7 Knowledge Cc applies the meaning of past to family 
members 
5 75.0" 8.3 46 10.9 Knowledge Cc knows what present means 
6 7.7 87.7" 3.1 14 Knowledge Cc knows that group work depends on 
sharing ideas 
10 1.4 13.3 4.3 77.2" Knowledge G recognizes an example of change in 
a community 
| 
3 3.9 7.1 24 | 86.3" Process Skills L identifies an appropriate question to be 
asked in an interview 
7 5.9 14.3 76.0° 2.8 Process Skills L infers relationship between key life events, 
age, and the future 
11 6.2 75.7* 10.0 7.0 Process Skills L identifies an example of technological change 
over time 
12 45 8.9 85.3" 0.7 Process Skills L draws conclusions from a graph 
15 77.1" 8.6 6.1 7.5 Process Skills L synthesizes information from interviews 
16 8.7 8.7 75.0* 6.6 Process Skills M draws conclusions by inference from the 
symbols on a map 
8 77.6* 12.2 25 | 7.3 Process Skills A predicts how communities will look in the 
future 
9 22.0 8.4 8.6 60.4* Process Skills A predicts the type of communication that will 
be used in the future 
13 11 77.3" 2.4 18.2 Process Skills A draws conclusions about an individual’s 
contribution to the community 
14 1.3 1.7 84.4* 11.9 Process Skills A evaluates an individual’s contribution to the 


community 


*correct answer 


Category Legend: F—Facts 


L—Locating, Organizing, Interpreting Information 
C—Concepts 


M—Mapping 
G—Generalizations 
A—Analyzing, Synthesizing, Evaluating 
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Table 2-5B 
Grade 3 Social Studies 
Results for Individual Multiple-Choice Questions 
Topic B: Communities Need Each Other 


June 1992 
Distribution of Responses (%) 
Item Component Category Curriculum Standard 
A B c D 
19 89.0* 49 1.8 3.6 Knowledge Cc differentiates a want from a need 
21 10.6 6.4 69.5* 12.4 Knowledge Cc recognizes an example of a service 
22 46 8.9 75.0" 10.3 Knowledge Cc knows the difference between rural and 
urban goods 
23 3.1 14.8 77.7" 3.3 Knowledge Cc knows the difference between rural and 
urban goods 
31 47 73.9* 9.2 11.2 Knowledge Cc shows an understanding of the concept of 
exchange 
34 3.1 5.1 10.6 80.6* Knowledge Cc differentiates aspects of rural life from urban 
life 
20 3.0 6.7 5.5 81.5* Process Skills L identifies places where goods are made rather 
than where services are provided 
26 68.5* 4.8 15.0 10.9 Process Skills L locates information by inferring from a title 
29 2.3 2.7 4.0 90.1* Process Skills M interprets information on a map using its 
legend 
30 83.6* 7.0 4.7 3.9. Process Skills M uses cardinal directions to draw conclusions 
from a map 
18 2.3 6.2 89.0* 1.7 Process Skills A classifies an example of a need 
27 86.2" 2.0 1.6 3.3 Process Skills A relates geography to goods produced ina 
community 
32 21.8 6.4 4.7 66.4* Process Skills A synthesizes information provided in a short 
paragraph on exchange of goods 
33 10.7 81.1" 4.7 2.2 Process Skills A synthesizes information provided in a short 
paragraph on exchange of goods 
*correct answer 
Category Legend: F—Facts M—Mapping 
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L—Locating, Organizing, Interpreting Information 


C—Concepts 


G—Generalizations 
A—Analyzing, Synthesizing, Evaluating 


Table 2-5C 
Grade 3 Social Studies 

Results for Individual Multiple-Choice Questions 
Topic C: Special Communities 


June 1992 
Distribution of Responses (%) 
Item Component Curriculum Standard 
A B Cc D 

35 43 81.2" 8.9 5.0 Knowledge identifies a custom in a native community 

37 30.2 17.3 35.9" 15.9 Knowledge understands that wanting to learn about 
another custom is showing respect 

39 9.4 72.7* 7.2 9.1 Knowledge recognizes examples of Chinese customs 

43 1.7 1.8 90.1* 6.0 Knowledge recognizes examples of Chinese customs 

49 2.8 56.1" 26.5 14.0 Knowledge understands ways of showing respect 

36 10.1 7.0 8.1 74.3* Knowledge knows the concepts of tradition/customs 

40 25.8 41.1* 26.6 5.5 Process Skills distinguishes between fictional stories and 
factual information about China 

41 3.0 6.8 5.8 83.7" Process Skills identifies possible sources of information 
about special communities 

45 6.5 6.2 6.7 80.0* Process Skills draws conclusions about information in a short 
passage 

48 9.0 12.1 63.6" 14.6 Process Skills acquires information by reading 

46 8.5 8.0 79.7" 2.9 Process Skills uses intercardinal directions to interpret 
information on a map 

38 86.2* 4.7 5.5 2.7 Process Skills analyzes the ways people perpetuate 
their lifestyle 

42 1.8 11.7 80.5" 5.4 Process Skills analyzes the ways people perpetuate 
their lifestyle 

44 82.7* 3.4 9.4 3.9 Process Skills analyzes participation in traditions 

50 45 6.2 7.0 80.0* Process Skills analyzes ways people keep their customs 

“correct answer 
Category Legend: F—Facts M—Mapping 


L—Locating, Organizing, Interpreting Information 
C—Concepts 


G—Generalizations 
A—Analyzing, Synthesizing, Evaluating 
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Table 2-6 

Grade 3 Social Studies 

Results for Individual Mapping Questions 
June 1992 


Item Distribution of Responses (%) Curriculum Standard 


Topic A: My Community in the Past, Present, and Future 


17 Duck Lake* Muriel Lake Sinking Lake Kehewin Lake Process Skills Mapping uses cardinal 
65.6 6.6 1.9 1.0 directions to read 
amap 


Topic B: Communities Need Each Other 


| 
24 Exactly 4* Less than4 More than4 Other Responses | Process Skilis Organizing interprets and makes 
76.0 3.7 5.6 10.7 Information bar graphs 
25 North* South East West Process Skills Mapping identifies north on 
95.8 1.2 1.0 1.2 a globe 
28 Alberta* Saskatchewan Quebec Other Responses | Process Skills Mapping locates Alberta on 
93.2 1.7 1.5 2.3 a map of Canada 


Topic C: Special Communities 


47 . Fort Vermilion* Medicine Hat Edmonton Other Responses | Process Skills Mapping compares distances 
84.3 2.2 1.3 2.4 on a map 


*correct answer. 
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Reporting Categories compared with one another. Statistics presented in this table 


Rather, the results shown in are based on results achieved by 
Table 2-7 sh the total ks : 
nee a ar ae Table 2-7 can best be used in 29 941 students (those who wrote 
possible and the raw score : i : . 
averages for the reporting conjunction with parallel tables in the regular form and those who 
the jurisdiction and school reports. wrote Form F). 


te 7 . . . 
categories Variations in patterns of results 


can help indicate strengths and 
weaknesses in local educational 
programs. 


It is important to stress that the 
averages in the various reporting 
categories cannot be directly 


Table 2-7 

Grade 3 Social Studies 

Raw Score Results by Reporting Category 

June 1992 

Maximum Raw Score 
Possible Raw Score Standard 
Reporting Category Raw Score Average Deviation 

Topic A—My Community in the Past, Present, and Future 17 13.3 3.0 
Topic B—Communities Need Each Other 17 13.8 3.1 
Topic C—Special Communities 16 11.7 3.2 
Knowledge 
(Facts, Concepts, Generalizations) 
Topic A 6 4.9 1.3 
Topic B 6 4.7 1.6 
Topic C 6 4.1 1.5 
Process Skills: 
Locating/Organizing/Interpreting 12 8.9 2.3 
Geography and Mapping 8 6.7 1.5 
Analyzing/Synthesizing/Evaluating 12 9.5 2.2 
Major Components: 
Knowledge 18 13.6 3.5 
Process Skills 32 25.1 5.2 
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Examiner’s Observations 


Achievement Test 

Students achieved very close to or 
higher than the acceptable 
standard in both knowledge and 
process skills and on the total test. 
In all three areas, more students 
than expected met the standard of 
excellence. 


1. The BEST meaning of community is 


86.1 © a place where people live, work, 
and play together 
5.0 o a place where people do 
different things 
3.3 o a special way of doing things 
4.0 0 apiece of land or region 


26. Titles of books help us to find information. 
Which book would BEST help us find 
information about goods that are produced 
in Alberta? 


Cities of 


Alberta 


480 


Services in 
Alberta 


Farming 


in Alberta 
Today 


15.0 0 10.9 o 


36. Another word for custom is 


10.1 o community 
7.0 o language 
8.1 o respect 
74.3 @ tradition 


39. Sue's family is from China. Her mother 
often cooks food in a wok and the family 
eats using chopsticks. This is an example 
ofa 


9.4 o change 
72.7 @ custom 
7.2 0 community 
9.1 © contribution 


38. People know they are in a special 
community because of the 


86.2 e way people keep their customs 
4.7 o hospitals people use 
5.5 o crops people grow 
2.7 o cars people drive 
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Most students have learned the 
content of the course. Their 
achievement on the test shows a 
good understanding, which enables 
them to relate their knowledge of 
social studies concepts to the real 
world. In addition, most students 


apply problem solving and critical 
thinking skills within the context 
of Grade 3 Social Studies. 


A discussion of specific areas of 
strength and difficulty for Grade 3 
students follows. 


are proficient at the process skills 
expected in Grade 3. They can 


Acceptable Standard—Sample Questions and Commentary 


What did students know and do well? 


Generally, students achieving the acceptable standard or higher had a 
sound understanding of the knowledge and basic process skills in the 
Grade 3 Social Studies program. Specifically, these students had a good 
understanding and could apply their knowledge of basic concepts such as 
community, past, present, future, wants, needs, cardinal directions, and 
tradition. Many of these students were also able to interpret graphed 
information and to demonstrate mapping skills. Questions 1 and 12 are 
examples. The circle beside the correct answer has been filled in, and the 
number is the percentage of students choosing each alternative. 


Where did students have difficulty? 


Question 26 required students to make a connection between “goods 
produced” and “what is made.” Almost all students achieving the standard 
of excellence were able to do this. Many other students, however, linked 
production to service or to farming, and they therefore incorrectly chose 
alternatives C or D. 


Questions 36 and 39 required students to demonstrate a basic 
understanding of customs. Many students achieving the acceptable 
standard on the test overall were unable to do this. Interestingly, more 
students were able to relate “special communities” to “customs” in 
question 38. 


This graph shows why people moved to Fox River. 
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Good Find Buy Play 
Weather Jobs Land Sports 
Reasons 


12. Why did most people move to Fox River? 


4.5 o To enjoy good weather 
8.9 o To find jobs 

85.3 e To buy land 

0.7 o To play sports 


| 


Use the information below to answer 
questions 32 and 33. 


Lumber is shipped from Vancouver to Regina 
on big trucks. People in Regina buy this lumber 
to build their houses. Wheat is shipped from 
Regina to Vancouver where it is used to make 
bread. In this way, the people in both cities get 
the things they need. 


32. What is this paragraph saying about 
exchanging goods? 


21.8 o Lumber is used in Regina to 
build houses. 
6.4 © Trucks are needed to transport 
wheat. 


4.7 o Cities use many different services. 


66.4 e Communities depend on each 
other for goods. 


13. When Mr. Hill moved to Fox River, he 
opened a small bakery. What contribution 
does Mr. Hill's bakery make to the 
community? 


1.1 © It provides a place to play sports. 
77.3 @ It produces goods for people. 
2.4 o It causes pollution. 

18.2 © It provides many jobs. 


40. In which book could you read FICTION 


about China? 


25.8 o Places in China 

41.1 @ Chinese Fairytales 

26.6 o The Encyclopedia of China 

5.5 o A Dictionary for Elementary 
Students 


42. The children in these pictures keep their 
traditions by 


1.8 o wearing jeans to school 

11.7 © speaking a different language 
80.5 e celebrating in special ways 
5.4 o eating special food 


Standard of Excellence—Sample Questions and Commentary 


What did students know and do well? 


Students achieving the standard of excellence had little difficulty handling 
questions that involved detailed reading passages and the use of high-level 
process skills such as analyzing and synthesizing. 


Question 32 required students to analyze the information in the 
paragraph to determine the main point. Question 42 required students to 
analyze both the visual and written information in the four separate boxes 
to draw a conclusion. Of students achieving the acceptable standard, 
77.1% were able to do this, as were 98.6% of students achieving the 
standard of excellence. 


To successfully answer question 13, students needed to know that a 
bakery produces breads, cakes, and other products, and that these are 
examples of “goods.” Although students achieving the standard of 
excellence were able to make this connection and realized that a “small” 
bakery would not provide “many jobs,” other students were attracted by 
choice D. 


Where did students have difficulty? 


Students achieving the standard of excellence had difficulty with 
question 40, which required them to distinguish between fictional stories 
and factual information. Other than this, these students had almost no 
difficulties with Grade 3 Social Studies knowledge or process skills. 


Use the information below to answer question 42. 


On Chinese New Year, Chang goes to the Susan is ready for the powwow. She likes to do 
Dragon Parade. Later his family shares gifts her native dance for her friends there. 
and eats moon cakes. 


lan likes to wear a kilt and play the bagpipes at | Mary likes to wear the Polish dress that her 


Scottish parties. He also enjoys the Scottish mother gave her for Christmas. She also likes 
food his father makes. to sing and dance to Polish music. 
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Participation Skills Study participation skills. This study was multiple-choice achievement test 


General Description designed to answer the question also made judgments regarding the 

Tn wddifion tothe traditional “At what level are Grade 3 participation skills of students 

multiple-choice achievement test, students performing in participation § performing at Grade 3 and beyond 

Grade 3 teachers and 1 798 skills?” The teachers used three Grade 3 expectations. The 

students from a sample of 63 descriptors to rate each of their assessment criteria and the results 

schools throughout the province students in 11 areas of group of this study are shown below. 
interaction. 


were involved in an assessment of 


Teachers who set standards for the 


Descriptors for Rating Scale 


Occasionally, if ever —the student occasionally, if ever, demonstrates this behavior 
Often —the student demonstrates this behavior more than half the time, but not consistently 
Consistently —the student consistently demonstrates this behavior and very rarely does not 


Table 2-8 
Grade 3 Social Studies 
Participation Skills: Percentage Distribution of Ratings 


Rating Scale 
Group Interaction Occasionally, if ever} Often | Consistently 
1. Works co-operatively with a partner 11.0 44.6 44.4 
2. Demonstrates willingness to take turns in discussions and play 11.8 43.8 44.4 
3. Understands the need for group rules and the need to follow the rules ~ 9,0 42.2 48.8 
4. Attempts to encourage and offer approval to those he/she works with 23.3 46.0 30.7 
5. Participates co-operatively in group work 13.2 43.3 43.4 
6. Makes meaningful contributions to discussions, supporting ideas with facts and reason 20.6 44.2 35.2 
7. Takes part in making the rules for group work 14.8 48.9 36.3 
8. Accepts the role of leader and follower as the situation requires or demands 19.9 48.4 31.8 
9. Demonstrates respect for the rights and opinions of others 12.0 43.0 44.9 
10. Participates in a small group discussion or activity by following established rules 12.8 45.1 42.1 
11. Stays on task 16.9 41.9 42.2 
Table 2-9 
Grade 3 Social Studies 
Participation Skills: Students Achieving Standards 
Student Achievement Standards Percentage of Students 
Beyond Grade 3 Students were rated “Consistently” in 7 of the 11 skills 30.7 
At Grade 3 Students were rated “Often” or “Consistently” in 10 of the 11 skills but had fewer 36.0 
than 7 “Consistently” ratings 
Not Yet At Grade 3 Students were rated “Often” or “Consistently” in fewer than 10 of the 11 skills 33.3 
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Examiner’s Observations 


Participation Skills 

Students were asked to complete 

a self-assessment of their 
participation skills using the 
descriptors “Often,” “Sometimes,” 
and “Not Often.” Fewer than 50% 
indicated that they “Often” praise 
group members who have good 
ideas. A similar percentage 
indicated that they do not “Often” 
demonstrate good leadership skills. 
Yet, the students indicated a very 
positive perception of their 
participation skills. For example, 
over 70% of the students perceived 
that they “Often” do their share of 
the work and “Often” follow the 
rules of the group. 


The instrument used by both the 
teachers and the students provided 
a limited amount of information 
regarding participation skills, 
although the Grade 3 teachers who 
participated in the study were 
pleased with the initiative and 
enthusiastic about the general 
trend to assess a broader range of 
skills as defined in the Program of 
Study. More work is needed on the 
design and administration of the 
instrument. 


Relationship between 
Students’ Participation Skills 
and the Achievement Test 
Generally, there is a direct and 
positive relationship between a 
student’s attainment of 
participation skills and a student’s 
performance on the achievement 
test. The results of the achievement 
test indicate that 56.4% of students 
who are performing below Grade 3 
in participation skills are also 
below the acceptable standard on 
the achievement test, whereas 
62.8% of students who are above 
Grade 3 in participation skills also 
met the standard of excellence on 
the achievement test. Students 
meeting each standard on the 
achievement test performed well. 
Although 85.6% of all students met 
or exceeded the acceptable 
standard on the achievement test, 
70.6% of students sampled 
demonstrated at or above Grade 3 
participation skills. 
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Contexts for Learning Study study designed to identify and Studies. Teachers reported on the 
General Description examine relationships among types and frequency of use of 
In June 1992, 104 Grade 3 Social various contexts for learning and instructional strategies, writing 


: their effect on student achievement. activities, classroom resources, and 
Studies teachers from 59 schools ‘ Sale Re 3 : 
d 264 students f th Students responded to questions other activities in their social 
ae BLUMenite srOMLACrOse Une related to the attitudes outlined in studies classes. 


province participated in a pilot 


the Program of Studies for Social 


Table 2-10 
Grade 3 Social Studies 
Contexts for Learning: Percentage Distribution of Student Responses 


No Not 
Question response| No | Maybe | Yes sure 
1. ls someone wrong when two people can’t agree? 1.9 38.3 23.9 18.9 17.0 
2. Can you learn from other students? Ww 9.1 11.7 76.1 1.9 
3. Are you interested in other people's ideas? 11 13.6 22.7 55.7 6.8 
4. Can you learn from people who are different from you? 1.1 7.2 9.5 78.4 3.8 
5. Do you want to listen to what someone else has to say if their opinion is different from yours? 1.9 11.4 14.8 63.3 8.7 
6. Is it okay if some people have opinions that are different from yours? | 4.2 4.2 86.4 4.2 
7. Do you ever make fun of other students? 11 §5.3 19.3 18.9 5.3 
8. Do you like working with students who are different from you? 1.5 8.3 11.7 69.7 8.7 
9. Do you like to learn about special communities? 0.8 8.0 8.0 77.3 6.1 
10. Can you get good ideas from people who live in special communities? 1.9 3.0 9.5 78.8 6.8 
11. Should people in special communities be more like everyone else? 19 61.7 8.0 15.2 13.3 
12. Can students like you do anything to make their community better? 1.9 9.5 14.8 63.6 10.2 
13. Should students like you try to make their community better? 1.9 8.7 10.2 70.5 8.7 
14. Do you like learning about the past of your community? 11 6.4 8.3 79.2 4.9 
15. Should all students know about their community's past? 11 13.6 20.8 53.4 11.0 
Table 2-11 
Grade 3 Social Studies 


Contexts for Learning: Percentage Distribution of Teacher Responses 


No Less than Once a Several times Every 
Question response Never once a week week a week day 
1. How often did you use the following 
instructional strategies with your social studies 
students this year? 
a. Full class discussion _— _ 1.0 9.6 66.3 23.1 
b. Reading from a textbook 2.9 1.0 34.6 23.1 34.6 3.8 
c. Worksheets — 1.9 1.9 41.3 29.8 25.0 — 
d. Small group discussion _— 1.0 24.0 29.8 40.4 48 
e. Library research 1.9 6.7 67.3 15.4 7.7 1.0 
f. Viewing films or videos —_— _— 74.0 19.2 6.7 — 
g. Oral presentation — 1.9 62.5 22.1 10.6 2.9 
h. Project work 2.9 1.0 51.9 26.9 13.5 3.8 
Continued 
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Table 2-11 (continued) 


No Less than Once a Several times Every 
Question response once a week week a week day 
2. How often did your class do the following 
kinds of writing in social studies this year? 
a. Journal/diary entries 1.9 33.7 17.3 16.3 2.9 
b. Stories = 59.6 26.9 5.8 1.0 
c. Writing about a personal experience 1.0 57.7 24.0 6.7 2.9 
d. Copying notes from the board 3.8 51.9 21.2 11.5 _ 
e. Ideas from research 2.9 44.2 33.7 14.4 1.0 
f. Ideas from class/group discussion 1.0 15.4 33.7 43.3 5.8 
Question/Statement No response Yes No 
3. Were the following activities a part of the Grade 3 
social studies program this year? 
a. Field trips 1.0 93.3 5.8 
b. Simulation, role play, co-operative games 1.0 83.7 15.4 
c. Listening to guest speakers 1.0 81.7 17.3 
d. Special celebrations _ 84.6 15.4 
e. Problem solving/decision making _ 99.0 1.0 
f. Writing assignments 1.0 98.1 1.0 
g. Completing handouts _ 96.2 3.8 
h. Answering prepared questions 1.0 92.3 6.7 
i. Research projects 1.0 90.4 8.7 
j. Making models and/or puppets 1.0 63.5 35.6 
k. Making maps and/or charts 1.0 98.1 1.0 
|. Making posters and/or books 2.9 88.5 8.7 
4. This year, the following items were readily available 
in my classroom for my students or for my use in 
planning and instruction. 
a. Atlas 1.0 84.6 14.4 
b. Dictionary _ 99.0 1.0 
c. Encyclopedias 48 57.7 37.5 
d. Globe 1.0 84.6 14.4 
e. Newspapers 2.9 57.7 39.4 
f. Magazines 1.9 73.1 25.0 
g. Travel brochures 5.8 41.3 52.9 
h. Films, videos 1.9 90.4 7.7 
i. Maps = 99.0 1.0 
j. Teacher Resource Manual 1.0 95.2 3.8 
k. Recommended resources 11.5 78.8 9.6 
5. This year, the following activities were carried out by the 
Grade 3 class as a part of the social studies unit of study. 
a. Writing a letter to a newspaper 9.6 71.2 19.2 
b. Cleaning up an outdoor area 7.7 6.7 85.6 
c. Recycling projects _ 85.6 14.4 
d. Collecting items to help people in communities meet 
their needs 1.0 80.8 18.3 
e. Fundraising 6.7 43.3 50.0 
f. Writing a letter to decision makers 5.8 38.5 55.8 
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Examiner’s Observations 


Student Attitude Questionnaire 
—Contexts for Learning 

On the student questionnaire, 
students indicated that they have a 
marked tolerance and openness to 
people different from themselves 
and to communities other than 
their own. A large majority of 
students are accepting of differing 
opinions and indicate a willingness 
to learn from others. 


Student responses were lowest in 
the area of acceptance. For 
example, to the question “Do you 
ever make fun of other students?”, 
18.9% of the students answered 
“yes.” This response indicates that 
students are not always kind to 
others. 


Relationship between Student 
Attitudes and the Achievement 
Test 

Students who met or exceeded the 
acceptable standard on the Grade 3 
Social Studies Achievement Test 
appear to have positive attitudes in 
social studies too. 


A definite pattern is found in 13 

of the 15 attitudinal questions. 
Students achieving at the standard 
of excellence responded more 
positively to all questions than did 
students achieving at the acceptable 
standard, who in turn responded 
more positively than did students 
who did not meet any of the 
standards set on the achievement 
test. For example, to the question 
“Is something wrong when two 
people can’t agree?”, 47% of 
students achieving the standard of 
excellence said “no,” 42.5% of 
students achieving the acceptable 
standard said “no,” and 17% of 
students not meeting any standard 
said “no.” This may be an indication 
that students who reach a higher 
level of achievement have positive 
attitudes. 
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The two exceptions to this trend 
were found in questions “Do you 
like working with students who are 
different from you?” and “Do you 
like to learn about special 
communities?” In response to these 
questions, the students who did not 
meet standards on the achievement 
test had more positive responses 
and the students achieving the 
standard of excellence were less 
positive. 


Teacher Questionnaire— 
Contexts for Learning 

Teacher responses to the 
questionnaire indicate that they 
choose from a wide variety of 
instructional strategies to help 
students learn. Teacher responses 
were lowest in making use of the 
library and in the teaching of 
library skills. | 


Results of the teacher questionnaire 
show that there is a wide variety of 
social studies related activities at 
the Grade 3 level. The diversity of 
these activities ranges from field 
trips to letter writing, recycling 
projects, and simulations. 


Many types of writing are used in 
Grade 3 classrooms. Of these, 
teachers’ responses indicate that 
journals and diaries are the least 
common. It is also interesting to 
note that copying notes from the 
blackboard is not frequently done. 


Teachers use a wide range of 
learning activities that help 
students learn in Grade 3 
classrooms. Up to 93.3% of classes 
go on field trips. In addition, 
simulation, guest speakers, and 
special celebrations are 
experienced by over 80% of classes. 
63.5% of teachers surveyed involve 
their students in making models or 
puppets. Encyclopedias are 
available to 57.7% of classes and 
some specialized items, such as 


travel brochures, are available to 
41.3% of classes. 


Responses also indicate that 99% 
of classrooms involve students in 
problem solving/decision making 
activities. 


The broad range of instructional 
strategies and learning activities 
used by teachers are indicators 
that students are actively engaged 
in their learning of social studies in 
Grade 3. 


Relationship between 
Learning Environment and the 
Achievement Test 

In this discussion, classrooms 
where most students met or 
exceeded the acceptable standard 
will be referred to as high 
achieving classrooms and 
classrooms where few students met 
or exceeded the acceptable 
standard will be referred to as low 
achieving classrooms. 


It appears that students in 88% 
of high achieving classrooms 
receive more weekly classroom’ 
time for full class discussion than 
do students in low achieving 
classrooms. Students in high 
achieving classrooms are also 
provided with more opportunities 
to do library research than are 
students in low achieving 
classrooms. 


The frequency of story writing, 
however, is inversely related to 
achievement. Students in high 
achieving classrooms in social 
studies tend to write fewer stories 
than do students in low achieving 
classrooms. 


For three classroom activities—full 
class discussion, library research, 
and oral presentations—results 


indicate that students who are 
more directly involved in these 
learning acitivities tend to have 
higher achievement. For example, 
in classrooms where students 
actively obtain their writing ideas 
from research, achievement test 
results are significantly higher 
than are the results from classrooms 
where students do not research 
their own ideas but have topics 
assigned by their teacher. It is also 
interesting that fewer high 
achieving classrooms use oral 
presentations than do low 
achieving classrooms. 


Another interesting finding is the 
frequency of using various 
instructional strategies. In 31.3% 
of low achieving classes, full class 
discussion was used on a daily 
basis whereas only 17.2% of high 
achieving classes used this 
strategy. Of low achieving 
classrooms, 53.1% used full class 
discussion several times a week 
whereas 75.9% of high achieving 
classrooms used this strategy. 
These differences reflect that 
teachers are using different 
strategies to meet the different 
needs of their students. 
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Section 3 


Grade 6 
English Language Arts 


Introduction 

In 1992 a number of assessments 
were carried out in Grade 6 English 
Language Arts. The achievement 
test was administered to students 
province-wide. Listening and 
viewing skills were assessed with 
a sample of Grade 6 English 
Language Arts students. As well, 
a sample of Grade 6 English 
Language Arts teachers and a 
sample of Grade 6 English 
Language Arts students 
participated in a pilot study to 
examine the relationship among 
various contexts for learning and 
their effect on achievement. The 
results from all these assessments 
follow. 


Achievement Test 
General Description 


The Grade 6 English Language 
Arts Achievement Test was in two 
parts. Part A: Composition was an 
80-minute assignment consisting of 
a picture and a prompt for writing. 
The assessment was designed to 
encourage students to use the 
writing process. Part B: Reading 
was a 60-minute reading test 
consisting of 50 multiple-choice 
questions based on nine reading 
selections. 


Statistics for the total test and for 
the components are based on the 
results achieved by 33 717 
students: 30 952 wrote the regular 
form and 2 765 participated in the 
achievement-over-time study, as 
shown in Table 3-3. This section of 
the report answers the following 
questions: 


¢ How many Grade 6 students 
wrote each form of the test or 
were absent and exempt? 

What percentage of Grade 6 
students attained the acceptable 
standard or higher in English 
Language Arts according to 
provincial criteria? 

What percentage of Grade 6 
students attained the standard 
of excellence or higher in English 
Language Arts according to 
provincial criteria? 

What did Grade 6 students 
know and what could they do in 
English Language Arts? 

What parts of the English 
Language Arts curriculum 
caused Grade 6 students 
difficulty? 


Summary of Results 


Results In Relation to 
Standards 

Results show that 75.6% of 
students who wrote the test 
achieved the acceptable standard 
or higher and 9.5% achieved the 
standard of excellence or higher on 
the total test. These results were 
lower than expected. 


Average Score 

The average total test score was 
60.6%, with a standard deviation 
of 14.0. 


Content of the Test 


Part A: Composition provided the 
students with a picture and a brief 
explanation as a prompt for their 
writing. Students chose the format 
(narrative, letter, or diary/journal 
entries) that would allow them to 
do their best writing while using 
their imagination and background 
experience. This part of the test 
was scored in five categories: 
Content, Organization, Sentence 
Structure, Vocabulary, and 
Conventions. Content and 
Organization were weighted to 

be worth twice as much as each 

of the other categories. 


Part B: Reading questions were 
based on reading selections from 
fiction, non-fiction, drama, poetry, 
and visual media. These selections 
were chosen to reflect the interests 
and varying ability levels of 
students in Grade 6 classrooms. 
Canadian material was used 
extensively. 
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Test Blueprint 


The test blueprint for Part A: 
Composition shows the reporting 
categories (scoring guide), a 
description of the writing 
assignment, and the emphasis of 
communication that was assessed. 


Table 3-1 
Grade 6 English Language Arts 
Achievement Test Blueprint 


Part A: Composition 
dune 1992 
Reporting Category (Scoring Guide) Description of Writing Assignment Range of Marks 
Content* (Selecting details to achieve a purpose) 
Events and/or actions should be plausible and appropriate 
to the student's purpose for communicating. The students 
should be able to describe characters and settings that are 
appropriate within the context of terms of reference The writing assignment follows a writing 5—Excellent 
established by the student. prompt, both of which the teacher reads 4—Proficient 
; aloud to the students. The assignment 3—Satisfactory 
Organization* (Organizing details into a coherent whole) allows the student to select the format that | 2—Limited 
The student should be able to place events in a coherent will best fit his/her approach to the prompt. 1—Poor 
sequence. INS—tnsufficient 


Sentence Structure (Structuring sentences effectively) 
The student should be able to use a variety of sentence 
structures effectively in writing. 


Vocabulary (Selecting words and expressions correctly and 
effectively) 

The student should be able to use words and expressions 
effectively in writing. 


Conventions (Using the conventions of language correctly 
and effectively) 

The student should be able to communicate clearly in writing 
by adhering to appropriate spelling, grammar, punctuation, and 
capitalization. 


*These two categories are each weighted to be worth twice as much as each of the other three. 
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The test blueprint for Part B: 
Reading shows the distribution 
of questions according to the 
reporting categories and cognitive 
levels under which questions are 
classified. 


Table 3-2 
Grade 6 English Language Arts 
Achievement Test Blueprint 


Part B: Reading 
June 1992 
Question Numbers by Cognitive Level Total 
Number of Percentage 
Reporting Category Literal Inferential Evaluative Questions of Test 


1. Identifying the Organization of Ideas 
The student should be able to attend to 7,23,27 13,25,49 20,45 8 16 
and analyze elements of the author's 
organization such as sequence, purpose, 
comparison, cause/effect, and imagery 
in a reading selection. 


2. Analyzing Details 


The student should be able to attend to 1,4,5,9, 6,22,29, 24,32,47 20 40 
and analyze the interrelationship of the 10,11,14, 39,41,44, 50 
details in a reading selection. 26,42 46 

3. Associating Meanings 
The student should be able to associate 3,8,12,15, 16,34,35, 13 26 
meanings of words and expressions 28,30,38, 36,37 
in context and evaluate the appropriateness 40 


of the author's word choice. 


4. Synthesizing Ideas 
The student should be able to synthesize 
the information within the reading 
selection to construct meaning. The 
student should be able to synthesize ideas 
from the entire reading selection in order 
to deduce the main idea and to predict 
plausible outcomes or conclusions. 


17,18,19, 9 18 
21,33,43, 
48 


18 50 100 
36 100 100 


Number of Questions 


Percentage of Test 
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Student Participation 


In June 1992, principals reported a 
total population of 37 837 students 
in Grade 6. Table 3-3 presents the 
number and percentage 
distribution of students who wrote 
the Grade 6 English Language 
Arts Achievement Test, who were 
absent, and who were exempted. 


Table 3-3 
Grade 6 English Language Arts 
Student Participation 
June 1992 
Number Percentage 
Category of Students of Students 


Students Absent 1 526 4.0 


Students Exempted from Writing 2 594 6.9 


Categories of Exemption: 


1. Special Needs Students 987 2.6 
2. Course Was Not Taught This Term 13 0.0 
3. English as a Second Language Students 393 1.0 
4. Other (as approved by the Superintendent) 1201 3.2 
Total Principals’ Reported Population 
Test Day, June 1992: 37 837 100.0 
Grade 6 Enrolment: September 30, 1991 37 533 


* Provincial results are based on test scores achieved by all students who wrote the Grade 6 English Language Arts test, including 
Francophone students, French Immersion students, and students in other programs. 


**Regular Form refers to the unmodified 1992 test. The modified tests are described in Section 8 of this report. 


i as 3 Sn RC 
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Results in Relation 
to Standards 


Figures 3-1 and 3-2 and Table 3-4 
show the percentage of students 
achieving the acceptable standard 
or higher and the standard of 
excellence or higher on the total 


test and on components of the test. 


The levels of performance were 
lower than expected for both 
standards, with the exception of 
Reading Skills at the standard of 
excellence. 


Se Sosa see Sst Rr 


Figure 3-1 

Grade 6 English Language Arts 

Percentage of Students Achieving Acceptable Standard or 
Higher on the Total Test and on Components of the Test 
June 1992 


100 


Total Test 
YY Achievement Standard* 
ZA (See Appendix A) 
* the percentage of students in the province expected to meet the acceptable standard or 


higher 
** the percentage of students in the province who met the acceptable standard or higher 


Composition Reading Skills 


Actual Results** 


Figure 3-2 

Grade 6 English Language Arts 

Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test and on Components of the Test 
June 1992 


Total Test Reading Skills 


WY Achievement Standard* 
Vs 4) (See Appendix A) 


Composition 


Actual Results** 


* the percentage of students in the province expected to meet the standard of excellence or 
higher 
** the percentage of students in the province who met the standard of excellence or higher 
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Table 3-4 
Grade 6 English Language Aris 

Students Achieving Standards on the Total Test and on Components of the Test 
dune 1992 


Reporting Category 
Standard of Excellence or Higher 


Provincial 
Maximum | Assessment 
Possible Standard* 
(Raw Score) 


Students Achieving Assessment 
Standard or Higher 

Actual 

Number 


on Both Components 


Total Test*** 100 
Compostion Only 35 
Reading Skills Only 50 
Acceptable Standard or Higher 

Total Test*** 100 
Compostion Only 35 
Reading Skills Only 50 
Below Acceptable Standard N/A 


* The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the lowest score a 
student must achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. 


See Appendix A. 


** The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provincial Assessment 


Standard. See Appendix A. 


*** The Composition score is multiplied by 50/35 before being added to the Reading score so that both components are weighted equally. 


It should be noted that the actual 
percentages of students achieving 
standards on the total test (9.5% 
and 75.6%) are based on the 33 717 
students in the regular programs 
who wrote the test. 


If, however, the percentages are 
based on the total population 
reported by principals (37 837), the 
percentages achieving standards 
on the total test would be: 
8.5% achieving the standard of 
excellence or higher 
67.4% achieving the acceptable 
standard or higher. 
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If the percentages are based on the 
September 30, 1991, Grade 6 
enrolment (37 533), the 
percentages achieving standards 
on the total test would be: 
8.5% achieving the standard of 
excellence or higher 
67.9% achieving the acceptable 
standard or higher. 


It is emphasized that the above 
percentages, based on total 
population and enrolment figures, 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for would have achieved standards. 
The absence of information on 
these students is nonetheless 
problematic. 


The number of students achieving 
the acceptable standard or higher 
and the standard of excellence or 
higher for each jurisdiction was 
analyzed to determine whether 
achievement in jurisdictions was 
below, meeting, or exceeding 
provincial achievement standards. 
Jurisdictions classified as meeting 
provincial achievement standards 
were those for which the difference 
between the actual number of 
students and the expected number 
of students at or above standards 
was not statistically significant. 


A 95% confidence interval was 
used; this criterion means that 
differences are only reported when 
there is a 5% or smaller probability 
that a difference of that size could 
occur by chance. The results are 
reported in Table 3-5. The 
percentage distributions in 

the tables are based on 211 
Jurisdictions (including private 
schools). 


Table 3-5 shows that 70.6% of 
jurisdictions were considered to 
have met or exceeded the 


Provincial Achievement Standard 
for the total test at the standard of 
excellence. As well, 51.2% of 
jurisdictions were considered to 
have met or exceeded the 
Provincial Achievement Standard 
for the total test at the acceptable 
standard. The finding that many 
jurisdictions were unable to meet 
expectations is not surprising 
given that the percentage of 
students in the whole province 
achieving or exceeding standards 
was below expectations. 


Table 3-5 


Grade 6 English Language Arts 


Percentage Distribution of Jurisdictions* Meeting Achievement Standards on the Total Test 


and on Components of the Test 


June 1992 


Component 


Standard of Excellence or Higher 
Total Test 

Composition Only 

Reading Skills Only 


Acceptable Standard or Higher 
Total Test 

Composition Only 

Reading Skills Only 


Percentage Distribution of Jurisdictions 
Not Meeting Provincial Meeting Provincial Exceeding Provincial 
Achievement Achievement Achievement 
Standard (Per Cent) Standard (Per Cent) Standard (Per Cent) 

29.4 67.3 3.3 
12.3 79.2 8.5 

5.7 81.5 12.8 
48.8 50.7 0.5 
49.8 49.3 0.9 
56.9 43.1 0.0 


* Jurisdictions with fewer than five students are excluded, as the statistical significance of the difference between the number actually 
meeting or exceeding the standard and the number expected to meet or exceed the standard when calculated and reported is not 
educationally meaningful. 


Results for 
Part A: Composition 


Results for Part A: Composition are 
most clearly understood in the 
context of the assignment students 
responded to and in the context of 
the scoring guides. The average 
raw score was 21.3 out of a possible 
35, with a standard deviation of 
5.5. Complete scoring guides are 
available from the Student 


Evaluation Branch, Alberta 
Education. 


Scoring Reliability 

Although the papers were scored 
on a one-marker system, 195 
randomly selected papers were 
re-marked so that a second set of 
scores was available to confirm 
scoring consistency. Of the scores 
awarded on the second reading, 
92.2% were identical to the original 


score on the same scale or varied 
by only one point. The one-marker 
system produces results that are 
reliable for groups of 25 or more 
students. Achievement test scores, 
however, are less reliable for 
individual students. 
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The results presented in Table 3-6 
are best considered in terms of the 
percentage of students that 
markers judged to have presented 
work that was 3 (Satisfactory) or 
higher for any reporting category. 
It is possible to draw conclusions 


Statistics presented in Table 3-6 
are based on results achieved by 
33 717 students. 


about local program strengths and 
weaknesses by comparing local 
percentages of 3 (Satisfactory) or 
higher scores in each reporting 
category with the provincial 
percentages. 


Table 3-6 


Grade 6 English Language Arts 
Percentage Distribution of Scale Points by Scoring Category 
Part A: Composition 


June 1992 
Scoring Category 
Sentence 
Scale Point Content Organization Structure Vocabulary Conventions 


(2) Limited 23.1 24.5 19.1 14.6 22.6 
(1) Poor 3.3 3.9 3.9 1.6 4.8 


(INS) Insufficient 0.1 0.1 0.1 0.1 0.1 


SSS 


Results for It is important to stress that the Statistics presented in this table 
Part B: Reading averages on the various reporting are based on results achieved by 

: : categories cannot be directly 31 890 students (those who wrote 
Reporting Categories 


Table 3-7 shows the total marks 
possible and the provincial raw 
score results for the reporting 
categories of the multiple-choice 
‘portion of the Grade 6 English 
Language Arts Achievement Test. 
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compared with one another. 
Rather, the results shown in 
Table 3-7 can best be used in 
conjunction with parallel tables in 
the jurisdiction and school reports. 
Variations in patterns of students’ 
responses to questions can help to 
indicate strengths and weaknesses 
in local educational programs. 


the regular form and those who 
wrote Form M). 


Table 3-7 
Grade 6 English Language Arts 
Raw Score Results by Reporting Category 


Part B: Reading 
June 1992 
Number of Raw Score Raw Score 
Reporting Category Questions Average Standard Deviation 

Total 50 30.1 8.5 
identifying the Organization of Ideas 8 48 1.9 
Analyzing Details 20 12:7 3.7 
Associating Meanings 13 7.6 2.6 
Synthesizing Ideas 9 4.9 1.8 
Literal 12 8.6 2.5 
Inferential 20 11.6 3.7 
Evaluative 18 10.0 3.4 


Percentage of Students 


Choosing Each Alternative 


Table 3-8 shows the percentage of 
students who chose each 


alternative (A, B, C, and D) for 


each multiple-choice question. The 
correct response for each question 
is identified with an asterisk and 


the curriculum standard each 


x 


Table 3-8 


Grade 6 English Language Arts 
Results for Individual Multiple-Choice Questions* 


question measures is specified. 
The questions are grouped by 
reporting category. 


questions can help to indicate 
strengths and weaknesses in local 
educational programs. 


The results shown in Table 3-8 can Statistics presented in Table 3-8 


best be used in conjunction with 
similar tables in the jurisdiction 


are based on results achieved by 
31 890 students (those who wrote 


and school reports. Variations in the regular form and those who 
patterns of students’ responses to wrote Form M). 


June 1992 
Distribution of Responses (%) 
Item Category Curriculum Standard 
A B Cc D 
7 12.7 9.9 65.6" 11.5 ile) understands the basic purpose of setting in a play 
x 
13 10.9 54.9* 19.5 14.0 ile) infers the meaning of the imagery in a stanza of a poem 
20 13.2 54.9* 8.5 23.2 lo makes a judgment about a cause-and-effect relationship 
from the context of a sentence 
23 65.4* 8.0 6.7 19.5 fe} recognizes a cause-and-effect relationship directly 
stated in a passage 


* correct answer. 
“The sum of the percentages for each question may be less than 100% because the No Response category is not included. 


Continued 


Category Legend: |0—Identifying the Organization of Ideas AD—Analyzing Details AM—Associating Meanings Sl—Synthesizing Ideas 
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Table 3-8 (continued) 


Distribution of Responses (%) 


Item Category Curriculum Standard 
A B Cc D 
25 13.7 12.6 7.3 66.0* 10 understands the author's purpose in repeating a phrase 
throughout a passage 
27 12.9 7.2 17.3 62.2" lO infers a cause-and-effect relationship from information 
in a passage 
45 8.5 53.1* 18.5 17.7 lo makes a judgment about a cause-and-effect relationship 


from the context of a paragraph 


49 9.4 11.3 15.6 61.4" 10 infers the author's purpose for using capital letters for 
specific words in a poem 


1 2.8 17.4 75.3" 4.4 AD locates directly stated details of setting 
4 3.3 6.3 81.5* 8.8 AD locates directly stated details of character motivation 
5 2.5 88.3* 4.9 4.2 AD locates pertinent details related to motivation of 
the main character(s) 
6 9.5 28.1 57.4* 5.0 AD uses pertinent details to infer character motivation 
9 7.2 78.9* 9.1 4.5 AD locates directly stated details of character motivation 
10 9.7 4.0 76.2* 9.8 AD locates directly stated details of scene 
11 2.9 9.7 6.8 80.4" AD locates directly stated details of character motivation 
14 41 65.3* 4.8 25.6 AD locates directly stated details of setting in a sentence 
22 43.1" 16.3 16.7 23.3 AD uses pertinent details to infer physical location stated 
in a passage 
24 6.6 30.5 50.6" 12.1 AD uses pertinent details to make a judgment about the 
main character(s) 
26 11.6 21.7 13.0 52.9* AD locates directly stated details about places in the passage 
29 7.8 5.1 81.9" 46 AD uses pertinent details to inter motivation of the minor 
character(s) 
32 16.3 11.2 61.7* 10.3 AD uses pertinent details to make a judgment about the 
setting of a poem 
39 8.9 12.7 60.0* 17.4 AD interprets a metaphor by relating pertinent details 
41 19.5 21.8 36.5" 21.0 AD uses pertinent details to infer motivation of the major 
character(s) 
42 11.6 77 14.0 65.4" AD locates directly stated details about physical geography 
44 17.3 30.9 15.4 34.2* AD uses pertinent details to infer motivation of the main 
character(s) 
46 18.4 11.3 10.5 57.4* AD uses pertinent details to infer motivation of the main 
character(s) 
“correct answer. Continued 


Category Legend: |O—Identifying the Organization of ideas AD—Analyzing Details AM—Associating Meanings Sl—Synthesizing Ideas 
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Table 3-8 (continued) 


Distribution of Responses (%) 


ttem Category Curriculum Standard 
B Cc D 

47 6.2 52.2* 14.0 25.1 AD makes a judgment about how pertinent details of the 
story are interrelated 

50 77 73.7* 9.3 7.0 AD uses pertinent details to make a judgment about print 
format 

3 16.8 9.7 14.7 58.6" AM infers the meaning of a word from its context in a sentence 
8 43.9" 31.4 16.6 8.0 AM infers the meaning of an expression from its context 

in a passage 

12 86.1" 8.3 1.4 4.0 AM infers the meaning of a word from its context in 
surrounding sentences 

15 10.4 50.2* 21.2 18.1 AM infers the meaning of an expression from its context 
ina stanza 

16 11.6 18.4 3.4 66.5* AM judges the most likely meaning of a word from its 
context in a stanza 

28 52.1* 18.5 18.8 10.1 AM infers the meaning of an expression from its context 
in a sentence 

30 11.8 64.7* 8.4 14.5 AM infers the meaning of a word from its context in a sentence 

34 17.4 51.4* 6.8 23.9 AM judges the most likely meaning of a word from its context 
ina poem 

35 12.3 12.2 5.2 69.7* AM judges the most likely meaning of an expression from 
its context in a poem 

36 76.6* 12.9 6.0 4.0 AM judges the most likely meaning of an expression from 
its context in a poem 

37 30.8 12.6 50.6* 5.4 AM judges the most likely meaning of an expression from 
its context in a poem 

38 44,5* 15.8 31.3 7.4 AM infers the meaning of a word from its context in the 
surrounding sentences 

40 16.9 8.6 24.5 49.0* AM infers the meaning of a word from its context in the 


surrounding sentences 


2 93.0" 1.2 4.0 1.7 S| synthesizes information from a sentence to draw a 
conclusion 
17 56.8* 13.4 4.7 24.9 SI synthesizes information to draw a conclusion about 


events in a selection 


18 15.5 42.3" 34.5 7.6 $l synthesizes information about character motivation from 
an entire selection 


19 14.8 18.3 56.9* 9.7 Sl synthesizes information in a passage to judge the meaning 
of a metaphor 


21 9.7 33.9 9.7 46.5* iS] makes a judgment about character motivation based on 
synthesis of information within a selection 


31 16.6 65.3* 7.4 10.1 SI synthesizes information from a passage to infer 
character motivation 


*correct answer. Continued 


Category Legend: 10—Identifying the Organization of Ideas AD—Analyzing Details AM—Associating Meanings Si—Synthesizing Ideas 
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Table 3-8 (continued) 


Distribution of Responses (%) 
Item Category Curriculum Standard 
A B Cc D 
33 13.4 14.7 §3.3* 18.2 Sl draws a conclusion about characters based on the 
synthesis of information from an entire selection 
43 30.8* 18.9 40.2 8.8 SI makes a judgment about a plausible outcome based on 
synthesis of information from an entire selection 
48 19.8 12.3 49.7" 15.7 Sl synthesizes information from an entire selection to 
judge the author's purpose 


*correct answer. 


Category Legend: 1O0—ldentifying the Organization of Ideas AD—Analyzing Details AM—Associating Meanings Si—Synthesizing Ideas 


Examiner’s Observations 


Achievement Test 

Although the overall quality of 
writing in English Language Arts 
in 1992 is quite comparable to that 
done in 1988, not enough students 
are meeting the acceptable 
standard. Those teachers who 
marked the tests were disappointed 
with many of the papers. They 
observed specifically that students 


1. The Wawaniki lived 


2.8 A. inthe forest 

17.4 B. inthe far north 

75.3*C. near the sunrise 

4.4 D. onthe southern seashores 


41. Terry's “symbolic act” (line 12) was to 


19.5 A. complete a personal challenge 

21.8 B. continue to believe in his dream 

36.5*C. dip his artificial leg into the Atlantic 
Ocean 

21.0 D. raise one dollar from each Canadian 
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are weak in the following areas: 
conventions (mechanics and 
grammar), sentence structure 
(grammatical correctness and 
variety), and conclusions 
(appropriate closure for the 


papers. They were lively, 
imaginative, and technically very 
well written. The quality of these 
papers far exceeded the 
expectations for students in a 
Grade 6 classroom. 


writing). The markers agreed that 


students showed real strength in 
the areas of introduction and 
vocabulary. A small number of 


A discussion of specific areas of 
strength and difficulty for Grade 6 
students follows. 


students wrote clearly excellent 


Acceptable Standard—Sample Questions and Commentary 


Question 1 required students to locate or identify the setting of a 
passage. Students achieving the acceptable standard can readily do 
this. 


Question 41 required students to infer the motivation of the main 
character. Students achieving the acceptable standard had difficulty 
with character motivation that was not directly stated. 


The strengths of students who demonstrated an acceptable standard of 
performance include an ability to 


* extract directly stated details of setting, scene, and character 
motivation (see question 1) 

* recognize cause-and-effect relationships when directly stated 

* recognize meanings of words and phrases from the context of the 
material 


However, many of these students had difficulty 


® inferring meanings of words and phrases from abstract material 

* inferring character motivation 

* making a judgment about the main idea of a reading selection 

¢ making a judgment about plausible outcomes in a reading 
selection 

* understanding a poem 


46. William took comfort if “he heard some 
slower unfortunate clomping behind” him 
(lines 24-25) because he 


18.4 A. felt sorry for the slower runners 
11.3 B. was ahead of some of the fast 


runners 

10.5 C. knew that he would cross the finish 
line first 

57.4*D. knew that he would not finish last in 
the race 


48. This article suggests that Terry’s dream 
was realized when 


30.8 *A. 


Canadians donated millions of 

dollars for cancer research 

18.9 B. he was the first to try to run across 
Canada 

40.2 C. Canadians became more aware of 

people with disabilities 

he became very famous 


8.8 D. 


Issues 


During the 1992 marking session, 
teachers were asked to comment 
on both parts of the test. Although 
many teachers were happy with 
the writing prompt and the choice 
allowed in writing format, they 
expressed concern about the 
80-minute time limit. In many 
cases, students’ “good copy” writing 
was not as well done as their 
“rough copy” writing, and at best 
was merely a recopying of the 


Standard of Excellence—Sample Questions and Commentary 


Question 46 required students to infer character motivation. 
Students achieving the standard of excellence had no difficulty making 


this type of inference. 


Question 43 required students to make a judgment about plausible 
outcomes based on their ability to synthesize information in the 
reading selection. A large number of students achieving the standard 
of excellence had difficulty doing this. Too many were drawn to 


alternative C. 


Students achieving the standard of excellence could 


¢ infer character motivation (see question 46) 
¢ infer meanings of words and phrases 
* make a judgment about the main idea of a reading selection 


However, many of these students had difficulty 


* interpreting highly abstract ideas and synthesizing the information 
* making a judgment about plausible outcomes in a reading selection 


(see question 43) 


original. A possible solution is to 
have students write only one 
legible copy and make any changes 
directly on that work. This would 
make the 80-minute time limit less 
of a concern. 


Another issue markers raised was 
that the written test did not 
accommodate some of the 
processes, such as brainstorming 
and group sharing, that are 
followed in the classroom. In 1993, 


we plan to allow students in groups 
of two to four about 10 minutes to 
discuss the assignment before they 
start writing. 


Finally, teachers suggested that 
students should do the reading and 
writing parts of the test on 
separate days. Starting in 1993, 
we expect to administer the 
Language Learning Achievement 
Test over two days. 
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Listening and Viewing 
Skills Test 


As they attempt to make sense of 
their learning experiences in all 
subject areas, students use 
language to explore, construct, 

and communicate ideas and 
information. The learning process, 
through its focus on meaning, 
enlarges the possibilities for the 
natural integration of all modes 
and strands of language, of which 
listening and viewing are integral 
parts. These connections are 
essential for effective and efficient 
language learning. Although the 
Language Learning component of 
the Program of Studies does not 
separate the “arts” of language into 
distinct strands, the achievement 


Table 3-9 


Grade 6 English Language Arts 


tests have historically concentrated 
on the reading and writing 
strands. Because listening and 
viewing are also critical parts of 
language learning, we wanted to 
assess these “arts” of language as 
well. 


To assess these skills, Grade 6 
teachers helped develop a listening 
and viewing skills test. The test 
was administered to a sample of 
565 students attending 15 schools 
in the province. Ten Grade 6 
English Language Arts lead 
teachers from across the province 
were involved in determining what 
performance on the listening and 
viewing skills test would reflect 
achievement beyond Grade 6 
expectations, achievement 


meeting Grade 6 expectations, and 
achievement not yet meeting 
Grade 6 expectations. 


General Description 


Listening Skills 

The listening skills component 
required the students to listen to 
three pieces of literature. After 
hearing the fiction and nonfiction 
selections, students answered some 
multiple-choice questions. After 
hearing the poetry piece, they 
retold what they heard. In 
addition, students listened to some 
directions and did what the 
directions asked them to do. Table 
3-9 presents the results of the 
listening skills component. 


Listening Skills: Distribution of Students in Relation to Grade 6 Expectations 


Student Achievement Number of Students Percentage of Students 
Students Who May Be Achieving Beyond Grade 6 17 3.0 
Students Who Are Achieving at Grade 6 439 77.7 
Students Who Are Not Yet Achieving at Grade 6 109 19.3 
Total 565 100.0 


Examiner’s Observations 
Listening Skills 

Students who have met Grade 6 
expectations are able to retell a 
great many significant details and 
events. Most of the time, the 
sequence of the events and/or 
details is correct. These students 
understand the denotative 
meanings of words they have 
heard; however, they may 
experience difficulty with the 
connotative meaning of words. 
Students who have met Grade 6 
expectations can paraphrase a 
spoken message but will experience 
some difficulty doing so. 
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They can also follow oral directions 
but will make a few errors in the 
process. These students, however, 
generally lack some of the 
confidence in their general 
listening skills typically found in 
students achieving beyond Grade 
6. As a result, their critical 
listening skills are not quite so 
well developed. 


Students who are achieving beyond 
Grade 6 expectations are 
consistently able to retell all 
significant details and events in 
proper order. These students 
understand both denotative and 


connotative meanings of words 
that they have heard. They can 
paraphrase a spoken message with 
no difficulty and follow oral 
directions with no errors. In 
addition, they display confidence 
and competence in their general 
listening skills. As a result, they 
are excellent critical listeners. 


Students who have not yet met 
Grade 6 expectations are usually 
able to retell only a few significant 
details and events. They have a 
confused idea of sequence. These 
students typically have a difficult 
time understanding all but the 
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most basic vocabulary. Connotative 
meanings are not understood at 
all. Paraphrasing for these 
students is difficult because they 
probably had considerable difficulty 
understanding what they heard. 


General Description 
Viewing Skills 

For the viewing skills component of 
the assessment, students were 


given two pictures. They had to 
describe what they saw in each 
picture and then compare and 
contrast how they felt when they 
looked at both pictures. In 
addition, they had to look at a 
cartoon and explain what message 
they got from it. 


The information collected was 
analysed by Grade 6 teachers. 


From the analysis, teachers were 
able to describe students’ 
performances that reflect 
expectations for Grade 6, beyond 
Grade 6, and not yet at Grade 6. 
Table 3-10 presents the results of 
the viewing skills component. 


Table 3-10 


Grade 6 English Language Arts 


Viewing Skills: Distribution of Students in Relation to Grade 6 Expectations 


Student Achievement 


Number of Students 


| Percentage of Students 


Students Who May Be Achieving Beyond Grade 6 33 


Students Who Are Achieving at Grade 6 


359 


Students Who Are Not Yet Achieving at Grade 6 173 


Total 


Examiner’s Observations 
Viewing Skills 

Those students who have met 
Grade 6 expectations are able to 
communicate in writing 
information they derive from a 
visual presentation; however, their 
statements tend to be less than 
specific. When asked to describe a 
picture, these students are able to 
pick out general details but the 
subtleties will escape their 
detection. They can compare and 
contrast different pictures only if 
the pictures are relatively 
straightforward and not too 
detailed. 


Students who are achieving beyond 
Grade 6 are able to communicate 
in writing information they derive 
from a visual presentation. The 
information is quite sophisticated 
because these students bring a 
broad general knowledge base to 
their understanding. When asked 
to describe a picture, these 


565 


students are able to pick out the 
most minute details. They also 
show an ability to compare and 
contrast different pictures and to 
provide many appropriate details. 


Students who have not yet met 
Grade 6 expectations demonstrate 
a limited understanding of what 
they view. Thus, they communicate 
in writing in only a superficial 
manner. When asked to describe a 
picture, these students pick out 
only the most obvious features. 
They are not able to compare and 
contrast different pictures unless 
the pictures are very basic. 


Relationship between 
Students’ Listening and 
Viewing Skills and the 
Achievement Test 

We were able to compare the 
results of 500 of the 565 students 
who participated in the listening 
and viewing assessment with their 
Grade 6 English Language Arts 


5.8 
63.5 
30.6 


achievement test results. The 
students who achieved or exceeded 
standards on the total achievement 
test did not, in most cases, have 
corresponding strengths in 
listening and viewing skills. 


Although 9.0% of the 500 students 
achieved the standard of excellence 
or higher on the achievement test, 
only 2.8% of these 500 students 
achieved beyond Grade 6 in 
listening skills and 6.6% achieved 
beyond Grade 6 in viewing skills. 
Similarly, a smaller percentage of 
students achieved Grade 6 
expectations on the viewing test 
(63.6%) than achieved the 
acceptable standard but not the 
standard of excellence on the 
achievement test (72.8%). 


In listening skills, more students 
achieved Grade 6 expectations 
(80.2%) than achieved the 
acceptable standard (72.8%) on the 
achievement test. 
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These results also indicate that 
listening and viewing skills are not 
strongly tied to each other. Only 
63% of students were classified in 
the same category on both 
listening and viewing components. 
None of the 565 students 
performed beyond Grade 6 
expectations on both components. 


That students’ viewing skills are 
weak supports some teachers’ 


Table 3-11 


Grade 6 English Language Arts 


comments that although students 
are generally exposed to a wider 
variety of visual stimuli than 
students in the past have been, 
they have not necessarily 
developed good visual 
discrimination skills. 


Survey on Attitudes Toward Language Learning 
Percentage Distribution of Student Responses 


Student Attitudes Toward 
Language Learning 
General Description 


In June 1992, 366 Grade 6 English © 
Language Arts students from 10 
schools in the province participated 
in a pilot study that surveyed 
students’ attitudes on various 
language learning issues. The 
results from the survey are 
presented in Table 3-11. 


sescaea 
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Neither 
No Strongly agree nor Strongly 
Statement response disagree Disagree disagree Agree agree 
1. To be a success in life, you 0.0 0.3 1.4 6.8 33.1 58.5 
need to be able to read. 
2. | think reading is an 0.0 0.3 0.0 2.5 38.3 59.0 
important skill. 
3. Everyone needs to know how 0.3 1.6 41 13.4 33.6 47.0 
to read. 
4, Out of all the activities | do in 0.0 10.1 22.1 32.2 20.5 15.0 
my academic subjects, | like 
reading best. 
5. | am a good writer. 11 5.2 13.1 33.6 37.4 9.6 
6. | enjoy reading my written 
work to: 
a. my classmates. 1.6 12.3 19.7 19.9 31.7 14.8 
b. other students in my 41 19.1 27.6 24.3 20.8 41 
school. 
c. teacher(s) and/or 3.3 13.1 23.8 25.4 26.8 7.7 
principal. 
d. other adults. 41 12.3 21.0 25.4 29.5 7.7 
e. my family. 2.7 6.6 5.5 13.1 37.4 34.7 
Continued 


Table 3-11 (continued) 


Neither 
No Strongly agree nor Strongly 
Statement response disagree Disagree disagree Agree agree 
ts ! enjoy having my writing 
“published”: 
a. in my classroom. 2.7 7.4 6.8 16.4 46.4 20.2 
b. in my school. 3.6 9.0 15.3 20.5 33.9 17.8 
c. in the community. 44 17.2 16.9 24.0 22.1 15.3 
8. | enjoy doing: 
a. book reports/reviews. 4.4 15.6 14.8 23.2 28.7 13.4 
b. movie reviews. 41 7.1 13.1 25.4 30.6 19.7 
9. In school | enjoy writing: 
a. reports 1.9 16.7 15.3 21.0 31.4 13.7 
b. essays. 3.0 19.1 24.9 30.6 18.3 41 
c. editorials. 4.4 18.3 21.6 39.6 13.9 2.2 
d. newspaper articles. 4.4 16.7 20.2 30.1 22.4 6.3 
e. journals/diaries. 3.3 13.9 14.8 18.3 35.0 14.8 
f. letters. 2.7 5.7 8.7 17.2 41.8 23.8 
g. _ stories. 1.4 49 5.7 14.8 37.2 36.1 
h. plays. 3.0 12.0 11.7 24.3 27.3 21.6 
i. poems. 3.3 16.7 16.1 16.9 29.0 18.0 
j. speeches. 7.7 24.9 21.0 21.9 14.8 9.8 
No 
Statement response Never Seldom Sometimes Usually Always 
10. | like to: 
a. talk about books | have 1.9 10.9 18.0 41.5 20.8 6.8 
read. 
b. talk about my favorite 1.9 28.7 33.1 23.0 7.7 5.7 
author(s). 
c. listen to literature being 3.3 25.4 21.3 23.5 16.1 10.4 
read. 
d. see movies based on 1.4 8.2 10.4 27.9 23.2 29.0 
books that | have read. 
e. read during my summer 2.2 15.8 13.4 20.2 20.2 28.1 
vacation. 
11. | choose my books based on the: 
a. author. 3.3 27.9 20.8 24.9 17.8 5.5 
b. type of reading material 1.9 7.1 5.7 16.1 28.4 40.7 
(e.g., mysteries, science fiction, 
fantasy). 
c. topic of reading material 1.6 7.1 11.7 26.5 29.5 23.5 
(e.g., animal stories, stories 
about characters the same 
age as me, stories about 
activities | like). 
Continued 
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Table 3-11 (continued) 


No 
Statement response Never Seldom Sometimes Usually Always 
12. In my leisure time | enjoy 
writing: 
a. reports. 2.7 49.7 28.1 14.2 3.8 1.4 
b. essays. 3.3 56.3 27.9 9.0 3.0 0.5 
c. editorials. 3.6 57.9 27.0 10.9 0.5 0.0 
d. newspaper articles. 4.9 52.5 23.5 13.9 3.8 1.4 
e. journals/diaries. 3.3 28.4 15.3 23.0 178 12.3 
f. letters. 3.0 13.7 14.5 28.7 24.9 15.3 
g. _ stories. 2.5 19.7 18.9 25.4 20.2 13.4 
h. plays. 41 44.8 20.5 15.8 7.4 7.4 
i. poems. 3.6 39.9 20.5 20.2 10.4 5.5 
j. | speeches. 8.7 53.6 20.8 10.7 3.0 3.3 
13. t enjoy going to: 
a. live theatre. 4.6 23.5 12.6 21.9 14.2 23.2 
b. concerts. 41 11.2 11.2 15.6 20.2 37.7 
c. symphony. 46 49.7 18.3 14.5 46 8.2 
d. opera. 6.3 55.7 14.2 11.2 5.2 74 
e. movies. 2.2 0.3 3.0 8.5 21.0 65.0 
f. museums. 3.6 12.3 13.9 25.4 21.9 23.0 
g. art galleries. 7.7 30.6 20.8 16.1 9.6 15.3 
14. | put my writing in a folder. 2.2 13.4 18.3 26.2 20.8 19.1 
15. | record my ideas and 2.5 51.6 20.8 18.6 46 1.9 
thoughts in a writing log. 
16. In my spare time: 
a. | read for my own enjoyment 11 10.9 12.0 21.0 22.7 32.2 
and pleasure. 
b. | read to get more information 2.2 12.0 21.0 35.8 18.0 10.9 
about something. 
c. | read to learn how to do 2.7 12.0 20.5 34.4 19.1 11.2 
something specific. 
17. ! use the school library to 
choose: 
a. free reading materials 0.8 4.4 6.8 16.7 38.8 32.5 
(e.g., novels, stories). 
b. information books and 2.7 11.5 17.8 29.8 23.8 14.5 
materials (e.g., encyclopedias, 
atlases). 
18. | use the public library to 
choose: 
a. free reading materials 2.7 11.7 12.0 16.7 23.8 33.1 
(e.g., novels, stories). 
b. — information books and 4.4 25.4 15.0 23.5 178 13.9 
materials (e.g., encyclopedias, 
atlases). 
c. records and/or tapes. 4.9 42.6 21.9 17.2 5.7 7.7 
d. videos. 6.6 48.1 18.9 12.3 6.6 7.7 
e. talking books. 5.7 63.9 15.3 8.2 3.3 3.6 
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Continued 


Table 3-11 (continued) 


No 
Statement response Never Seldom Sometimes Usually Always 
19. | read: 
a. magazines. 3.3 44 7.7 27.9 26.0 30.9 
b. newspapers. 3.3 10.9 16.9 32.2 18.3 18.3 
c. science fiction. 41 15.0 16.9 27.3 18.3 18.3 
d. fantasy. 3.8 14.5 15.3 27.3 20.2 18.9 
e. poetry. 3.3 31.4 24.6 22.1 10.7 7.9 
f. plays. 5.2 39.3 24.0 20.2 7.4 3.8 
g. biographies. 3.8 41.3 23.0 17.2 9.3 5.5 
h. _ historical fiction. 4.9 30.9 26.0 19.1 9.3 9.8 
i. mysteries. 3.8 8.5 6.6 22.7 23.0 35.5 
j. informational materials 4.9 22.7 27.6 24.0 12.8 7.9 
(e.g., encyclopedias, atlases). 
k. romance novels. 41 43.4 14.5 16.7 10.1 11.2 
|. comic books. 3.6 4.6 9.3 25.7 21.0 35.8 
m. labels on packages. 5.2 27.0 18.0 20.8 13.1 15.8 
n. — instructions. 3.3 13.9 16.7 20.5 19.1 26.5 
0. billboards. 3.0 21.3 18.6 27.0 14.8 15.3 
p. catalogues. 3.8 12.3 16.7 27.0 20.5 19.7 


Examiner’s Observations 


Student Attitude Questionnaire 
—Contexts for Learning 
Results indicate that the students 
surveyed overwhelmingly believe 
that everyone needs to know how 
to read, and over 97% say that 
reading is an important skill. 
Analysis of the student 
questionnaires indicates that the 
majority of students (69.1%) 
usually or always choose their 
leisure reading material by genre, 
and the most preferred reading 
materials are magazines, 
mysteries, and comic books. 
Students use school and public 
libraries mainly for choosing 
leisure reading materials. 

In both school and leisure time, 
students’ favored types of writing 
are journals/diaries, letters, and 
stories. Of these, letter-writing is 
most preferred during leisure time 
and story-writing is more popular 
in school. Many students keep 
their writing in a folder, although 
fewer than 7% regularly record 


their thoughts and ideas in a 
writing log. 


The majority of students enjoy 
having their writing “published” in 
the classroom (66.6%) or in their 
own school (51.7%). However, few 
students (37.4%) enjoy having their 
writing published in the greater 
community. This could indicate 
that in more familiar settings, 
where students know the 
expectations, they have greater 
confidence to take the risks that 
could accompany public display. 


The results of the survey also 
revealed that although students 
generally enjoy writing 
assignments in school, in their 
leisure time they choose writing 
activities less often than reading. 


Regardless of their writing ability, 
72.1% of students enjoy reading 
their written work to their 
families. However, fewer than 50% 
of students enjoy reading their 
written work to other people. This 


is a further reflection of students’ 
comfort with what they perceive as 
“safe” environments. 


Relationship between 

Student Attitudes and the 
Achievement Test 

We were able to compare the 
results of 320 of the 366 students 
who participated in the attitude 
survey with their Grade 6 English 
Language Arts Achievement Test 
results. Of the 33 students who 
achieved the standard of excellence 
or higher on the total achievement 
test, 60.6% liked reading better 
than any other activities done in 
the classrooms. This is sharply 
contrasted by students who 
achieved the acceptable standard 
but not the standard of excellence 
(N=234) on the total test or those 
who performed below the 
acceptable standard (N=53), where 
only 34.6% and 28.3%, respectively, 
liked reading better than other 
activities done in the classroom. 
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Students achieving or exceeding 
the standard of excellence on the 
total test tend to be good and 
enthusiastic readers. Examiners 
were not surprised to find that 
almost all students (93.9%) 
achieving the standard of 
excellence or higher indicate that 
they read for their own enjoyment 
and pleasure in their spare time, 
whereas the same is true for only 
58.1% of students at the acceptable 
standard but not the standard of 
excellence and 26% of those 
performing below the acceptable 
standard, Similar findings 
emerged from the questions related 
to reading for enjoyment during 
the summer, where 69.7% of 


students at the standard of 
excellence or higher, 51.5% of 
students at the acceptable 
standard but not the standard of 
excellence, and 30% of students 
below acceptable standard, 
responded affirmatively. 


Regarding leisure time reading, it 
is interesting to note that students 
who are below the acceptable 
standard read in order to learn 
something specific in their spare 
time more often (38%) than do 
students who are at the acceptable 
standard but not the standard of 
excellence (31.1%) or at the 
standard of excellence or higher 
(28.1%). This reinforces the belief 


held by many teachers that 
reading, especially for weaker 
students, must have a real purpose 
in order for students to actively 
engage in it. 


Teacher Questionnaire— 
Contexts for Learning 

In July 1992, 141 teachers from 
across the province who were 
marking the composition 
component of the Grade 6 English 
Language Arts Achievement test 
participated in a pilot study that 
surveyed the instructional 
strategies and learning 
environments they provided in the 
classroom. The results from the 
survey are presented in Table 3-12. 


Table 3-12 

Grade 6 English Language Arts 
Teacher Survey on Instructional Practices and Learning Environments 
Percentage Distribution of Teacher Responses 


Question 


How often did your students write 
in L.A. this year? 


How often did your students write 
in a journal for L.A. this year? 


How often did your students use 
the library in the school this year? 


How often did you set aside a 
special time to read to your 
students this year? 


How often did your students do 
silent reading (USSR) this 
year? 


No Less than Once Several times Every 
Response Never once a week a week a day day 
0.7 0.0 1.4 5.0 53.2 39.7 
0.7 11.3 21.3 19.1 33.3 14.2 
1.4 0.0 4.3 29.8 61.7 2.8 
1.4 0.7 8.5 11.3 43.3 34.8 
0.7 0.7 2.1 9.9 29.1 57.4 
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Table 3-12 (continued) 


No 
Question Response Never Rarely Sometimes Often Always 
6. This year, how often did your 0.7 0.0 0.7 23.4 63.1 12.1 
students have a choice about 
what they wrote? 
7. This year, how often did your 
students do the following before 
they wrote? 
a. Brainstorm 0.7 0.0 3.5 36.9 42.6 16.3 
b. Discuss in groups 0.7 1.4 6.4 37.6 42.6 11.3 
c. Doastory map 1.4 3.5 22.0 41.1 23.4 8.5 
d. Goto the library to do 1.4 0.7 12.8 46.1 37.6 1.4 
research 
e. Listen to the teacher 0.0 0.0 5.7 31.2 43.3 19.9 
f. | Watch a movie 0.0 5.0 52.5 36.9 5.7 0.0 
g. Goona field trip 0.0 12.1 58.2 27.7 2.1 0.0 
8. How often did your students use a 
computer at school to do 
the following this year? 
a. _ All of their writing (from draft 2.8 25.5 34.8 26.2 10.6 0.0 
to final copy) 
b. Some of the work for L.A. 0.0 12.1 36.2 36.2 15.6 0.0 
c. Editing their writing 0.0 23.4 31.9 32.6 12.1 0.0 
d. Checking their spelling 1.4 34.0 33.3 22.7 7.8 0.7 
e. Typing the final copy of their 0.0 14.2 25.5 42.6 16.3 1.4 
writing 
f. | Using a program for reading 0.0 48.9 28.4 19.1 3.5 0.0 
g. Using a program for spelling 1.4 44.7 31.2 16.3 5.0 1.4 
9. This year, how often did your 2.8 0.0 5.7 24.1 44,7 22.7 
students write a number of drafts 
before handing in the final product? 
10. This year, how often did you have 0.0 0.7 10.6 36.9 44,7 7.1 
conferences with your students 
about their writing? 
11. How often did your students have 0.7 2.1 9.2 39.7 44.7 3.5 
their writing published in the 
classroom, school, or community 
this year? 
12. How often did your students do the 
following in L.A. this year? 
a. “Readers theatre” 1.4 19.9 29.1 40.4 8.5 0.7 
b. Drama 21 2.8 19.9 51.1 22.0 2.1 
c. View films or videos 0.7 0.0 17.0 61.7 19.9 0.7 
d. Puppet theatre 1.4 50.4 26.2 17.7 4.3 0.0 
e. Story telling 2.8 5.7 21.3 44.7 24.8 0.7 
f. | Chanting 2.1 42.6 34.0 17.7 2.8 0.7 
g. Debates 1.4 5.7 36.9 41.1 14.9 0.0 
h. Oral presentations 0.7 0.0 2.1 36.2 51.8 9.2 
i. Role playing 0.0 0.7 13.5 50.4 31.9 3.5 
Continued 
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Table 3-12 (continued) 


No 
Question Response Never Rarely Sometimes Often Always 
13. | know that my students were 
using their writing skills in the 
following subject areas: 
a. Science 2.1 0.0 11.3 32.6 39.0 14.9 
b. Social Studies 1.4 0.0 2.1 12.1 52.5 31.9 
c. Math 2.8 9.9 34.0 36.2 13.5 3.5 
d. Physical Education 12.1 44.0 33.3 9.2 0.7 0.7 
e. Health 3.5 0.7 12.1 48.2 29.8 5.7 
Statement No response Yes No 
14. This year, the following materials 
were available in the classroom 
for the students: 
a. Magazines 0.7 85.1 14.2 
b. Newspapers 0.7 78.0 21.3 
c. Dictionaries 0.7 98.6 0.7 
d. Thesaurus 0.7 87.9 11.3 
e. Encyclopedias 2.1 68.1 29.8 
f.  Atlas/Globe 1.4 94.3 4.3 
g. Computer 2.1 51.8 46.1 
h. Poetry 2.1 89.4 8.5 
i. Fiction books 1.4 95.7 2.8 
j. | Non-fiction books 1.4 93.6 5.0 
k. Student-published books 5.0 68.1 27.0 
15. This year, | took my students on 
field trips to: 
a. _ Live theatre 6.4 58.9 34.8 
b. Concert 12.8 36.2 51.1 
c. Symphony 16.3 21.3 62.4 
d. Opera 19.1 19.9 61.0 
e. Movie 19.9 19.1 61.0 
f. Museum 14.2 46.8 39.0 
g. Artgallery 25.5 19.9 54.6 
16. My students had a portfolio or 0.0 94.3 5.7 
writing folder with samples of their 
writing. 
17. My students were writing for 0.7 85.8 13.5 
different audiences (e.g., younger 
students, peers, teachers, 
community). 
No Less than More than 
Statement response 1 hour 1-2 hours 2-4 hours 4 hours 
18. My students spent the following 1.4 2.8 29.8 44.7 21.3 
amounts of time reading in the 
content areas each week: 
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Examiners learned that 96.4% of 
the teachers surveyed provide 
silent reading (USSR) at least once 
per week. Classroom environments 
tend to be “print rich” with 
magazines, newspapers, both 
fiction and non-fiction books, as 
well as reference materials; 
although classes of high and 
medium achieving students tend 
to have of these resources slightly 
more than do classes of low 
achieving students. 


It was interesting to see that 78.1% 
of teachers responding to the 
survey indicated that they set 
aside special time to read to their 
students several times per week or 
even every day. Only 26.5% of 
students indicated that they 
usually or always like to listen to 
literature being read. 


Relationship between the 
Learning Environment and the 
Achievement Test 

We matched the teachers who 
responded to the survey with the 
Grade 6 English Language Arts 
achievement test results of the 
students in their classrooms. For 
the following analysis we included 
only the 94 teachers for whom we 
had achievement test results for 12 
or more of their students. For 
discussion of this analysis, 
classrooms where at least 85% of 
students met or exceeded the 
acceptable standard are referred to 
as high achieving classrooms 
(N=34) and those classrooms where 
fewer than 75% of students met or 
exceeded the acceptable standard 
are referred to as low achieving 
classrooms (N=31). The remaining 
classrooms will be referred to as 
medium achieving classrooms 
(N=29). 


Students in 72.7% of high 
achieving classrooms use the 
library several times a week or 
every day, whereas only 56.7% of 
low achieving classrooms use the 
library with the same frequency. 
In 78.8% of the high achieving 
classrooms, students often or 
always listen to the teacher before 
beginning writing. The same is 
true for only 60% of students in low 
achieving classes. Teachers of high 
achieving classes also have their 
students write multiple drafts 
more often than do teachers of 
medium and low achieving classes. 


All teachers surveyed give their 
students a choice about what they 
write, but teachers of high 
achieving classes give their 
students a choice less often (66.6%) 
than do teachers of medium 
achieving classes (85.8%) or low 
achieving classes (76.7%). Some 
teachers reflected that this might 
be expected since students write 
with greater confidence when 
assignments are clearly delineated. 
Other teachers noted that when 
repeatedly allowed to make their 
own writing choices, students tend 
to use the writing styles with 
which they feel most comfortable 
and therefore do not gain practice 
and experience with those less 
familiar. 
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Section 4 


Grade 9 
Mathematics 


Introduction 

In 1992 a number of assessments 
were carried out in Grade 9 
Mathematics. The achievement 
test was administered to students 
province-wide. Performance 
assessment tests were administered 
in a sample of Grade 9 Mathematics 
students. As well, a sample of 
Grade 9 Mathematics teachers and 
a sample of Grade 9 Mathematics 
students participated in a pilot 
study to examine the relationship 
among various contexts for 
learning and their effect on 
achievement. The results from all 
these assessments follow. 


Achievement Test 
General Description 


The Grade 9 Mathematics 
Achievement Test was divided into 
two parts. Part A had 40 multiple- 
choice questions each with four 
alternatives. Part B had 10 
numerical-response questions, 
which required students to 
calculate an answer and then 
record it on the answer sheet: 


The statistics for the total test and 
for the various components are 
based on the results achieved by 
27 888 students: 24 523 wrote the 
regular form and 3 365 participated 
in the achievement-over-time 
study, as shown in Table 4-2. This 
section of the report provides 
answers to the following questions: 


¢ How many Grade 9 students 
wrote each form of the test or 
were absent and exempt? 

¢ What percentage of Grade 9 
students attained the acceptable 
standard or higher in 
Mathematics to provincial 
criteria? 

* What percentage of Grade 9 
students attained the standard of 
excellence or higher in 
Mathematics according to 
provincial criteria? 

¢ What did Grade 9 students know 
and what could they do in 
Mathematics? 

¢ What parts of the Mathematics 
curriculum caused Grade 9 
students difficulty? 


Summary of Results 


Results in Relation 

to Standards 

Results show that 67.4% of 
students who wrote the test 
achieved the acceptable standard 
or higher and 8.9% achieved the 
standard of excellence or higher on 
the total test.. These results were 
lower than expected for the both 
standards. 


Average Score 

The average total test score was 
57.0%, with a standard deviation of 
19.8%. The average total raw score 
was 27.9 marks out of a possible 49, 
with a standard deviation of 8.2. 


Content of the Test 


The Grade 9 Mathematics 
Achievement test was designed to 
reflect the Grade 9 Mathematics 
curriculum standards. The scope of 
the test was limited to curriculum 
objectives that could be efficiently 
measured on a paper and pencil 
test. 


The test consisted of 50 questions 
in five content areas: Number 
Systems and Operations, Ratio and 
Proportion, Measurement and 
Geometry, Data Management, and 
Algebra. The questions were 
designed to measure achievement 
in four cognitive levels: Content, 
Knowledge, and Quantitative 
Literacy; Procedural Knowledge; 
Conceptual Understanding; and 
Problem Solving. These cognitive 
levels were, in turn, grouped under 
two major components: 

* Knowledge and Skills 

¢ Application and Problem Solving 
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Test Blueprint 


Table 4-1 presents the blueprint 
used to develop the Grade 9 
Mathematics Achievement Test. 
Classification of each question by 
component and cognitive level is 
indicated in the table. 


Table 4-1 
Grade 9 Mathematics 


Achievement Test Blueprint 
dune 1992 


and Test Emphasis 


* s—numerical-response question 
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Reporting Category 
Knowledge and Skills Application and Problem Solving Total Number 
Content, of Questions 
Curriculum Component Knowledge, and and Test 
Quantitative Procedural Conceptual Problem Emphasis 
Literacy Knowledge Understanding Solving 
Number Systems and Operations 1,2, 10 1s,* 2s 3, 4, 5, 6,7 8, 9, 5s 13 (26%) 
Ratio and Proportion 11, 16 4s 12, 13, 14 15, 3s, 6s 9 (18%) 
Measurement and Geometry 17, 22 18, 19 20, 21, 7s, 8s,9s| 23, 24 11 (22%) 
Data Management 29 25 26, 27, 28, 30 10s 7 (14%) 
Algebra 31 32, 33, 34 35, 36, 37, 38 39, 40 10 (20%) 
Total Number of Questions 9 (18%) 9 (18%) 21 (42%) 11 (22%) 50 (100%) 


Student Participation number and percentage Results for students in French 


In June 1992, principals reported a panes eas who wrote Immersion or See 

total population of 34 386 students te apis atte matics ghecconenn as neporied weparntely 

in Grade 9. Table 4-2 presents the Achievement Test, who were to participating schools and 
absent, and who were exempted. jurisdictions. 


Table 4-2 
Grade 9 Mathematics 
Student Participation 
June 1992 
Number Percentage 
Category of Students of Students 


Other Students Who Wrote: 1 481 4.3 


Francophone—Taught in French, Wrote in French 137 0.4 
Francophone—Taught in French, Wrote in English 0 0.0 
French Immersion—Taught in French, Wrote in French 1136 3.9 
French Immersion—Taught in French, Wrote in English 8 0.0 
Students Absent 1 671 4.9 
Students Exempted from Writing 3 346 9.7 


Categories of Exemption: 


1. Special Needs Students 1 881 5.5 
2. Subject Was Not Taught This Term 171 0.5 
3. English as a Second Language Students 183 0.5 
4, Language of Instruction Was Not English 4 0.0 
5. Other (as approved by the Superintendent) 1107 3.2 
Total Principals’ Reported Population 
Test Day, June 1992: 34 386 100.00 
Grade 9 Enrolment: September 30, 1991 35 430 


*Provincial results are based on test scores achieved by students who were in the “regular” Mathematics program. Such students are 
defined as those who were instructed in English and who wrote the English form of the achievement test, and those students who were 
instructed in a language other than English or French and who wrote the English form of the test. They are indicated by the shaded area of 
the table. 


™ Regular Form refers to the unmodified 1992 test. The modified tests are described in Section 8 of this report. 


***Results for Francophone or French Immersion students are reported directly to participating schools and jurisdictions and are not 
calculated in the provincial results. 


51 


Results in Relation to 
Standards 


Figures 4-1 and 4-2 and Table 4-3 
show the percentage of students 
achieving the acceptable standard 
or higher and the standard of 
excellence or higher on the total 


test and on components of the test. 


The levels of performance were 
lower than expected for both the 
standard of excellence and the 
acceptable standard. 
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Figure 4-1 \ 
Grade 9 Mathematics 

Percentage of Students Achieving Acceptable Standard or 
Higher on the Total Test and on Components of the Test 
June 1992 
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Application and 
Problem Solving 
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and Skills 


Actual Results** 


Y Achievement Standard* 
(A (See Appendix A) 


* the percentage of students in the province expected to meet the acceptable standard or 
higher 


** the percentage of students in the province who met the acceptable standard or higher 
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Figure 4-2 

Grade 9 Mathematics 

Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test and on Components of the Test 
June 1992 
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and Skills 
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Total Test 


Application and 
Problem Solving 


Actual Results** 


YY Achievement Standard* 
Y 


(See Appendix A) 


* the percentage of students in the province expected to meet the standard of excellence or 
higher 


** the percentage of students in the province who met the standard of excellence or higher 


Table 4-3 


Grade 9 Mathematics 
Students Achieving Standards on the Total Test and on Components of the Test 
June 1992 
Provincial Students Achieving Assessment 
Maximum | Assessment Standard or High 
Possible Standard* Expected ' Actual 
Reporting Category Score (Raw Score) Number Number 
Standard of Excellence or Higher 
Total Test 49*"* 42 4 183 2 469 
Application and Problem Solving Only 31 26 4 183 2971 
Knowledge and Skills Only 18 16 4183 3 545 
Acceptable Standard or Higher 
Total Test 49 23 23 705 18 807 
Application and Problem Solving Only 31 14 23 705 17 890 
Knowledge and Skills Only 18 9 23 705 21 085 
Below Acceptable Standard N/A N/A N/A 5 661 
on Both Components 


* The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the lowest score a 
student must achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. 


See Appendix A. 


“* The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provincial Assessment 


Standard. See Appendix A. 


*** Item 12 was removed from the test before scores were calculated. 


It should be noted that the actual 
percentages of students achieving 
standards on the total test for 
Grade 9 Mathematics (8.9% and 
67.4%) are based on the 27 888 
students in the regular programs 
who wrote the test. 


If, however, the percentages are 
based on the total population 
(34 386) reported by principals, 
less the French Immersion and 
Francophone students who wrote 
the test (1 481), the percentages 
achieving standards on the total 
test would be: 
7.5% achieving the standard of 
excellence or higher 
57.2% achieving the acceptable 
standard or higher. 


If the percentages are based on the 
September 30, 1991, Grade 9 
enrolment (35 430), less the French 


Immersion and Francophone 
students who wrote the test 
(1 481), the percentages achieving 
standards on the total test would 
be: 
7.3% achieving the standard of 
excellence or higher 
55.4% achieving the acceptable 
standard or higher. 


It is emphasized that the above 
percentages, based on the total 
population and enrolment, present 
the lowest estimate of achievement. 
It is highly likely that some of the 
students who were absent, exempt 
or not accounted for could have 
achieved standards. The absence of 
information on these students is 
nonetheless problematic. 


ee ee ed 


The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
jurisdiction was analyzed to 
determine whether jurisdictions 
were below, meeting, or exceeding 
provincial achievement standards. 
Jurisdictions classified as meeting 
provincial achievement standards 
were those for which the difference 
between the actual number of 
students and the expected number 
of students at or above standards 
was not statistically significant. 

A 95% confidence interval was 
used; this criterion means that 
differences are only reported when 
there is a 5% or smaller probability 
that a difference of that size could 
occur by chance. The results are 
reported in Table 4-4. The 
percentage distributions in 

the table are based on 189 
jurisdictions (including private 
schools). 
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Table 4-4 
Grade 9 Mathematics 
Percentage Distribution of Jurisdictions* Meeting Achievement Standards on the Total Test and 
on Components of the Test 
June 1992 
Percentage Distribution of Jurisdictions 
Not Meeting Provincial Meeting Provincial Exceeding Provincial 
Achievement Achievement Achievement 
Component Standard (Per Cent) I Standard (Per Cent) Standard (Per Cent) 
Standard of Excellence or Higher 
Total Test 29.6 68.8 1.6 
Application and Problem Soiving Only 25.9 71.4 2.7 
Knowledge and Skills Only 16.9 74.6 8.5 
Acceptable Standard or Higher 
Total Test 64.0 34.9 1.1 
Application and Problem Solving Only 68.3 31.2 0.5 
Knowledge and Skills Only 42.3 §7.2 0.5 


“Jurisdictions with fewer than five students are excluded, as the statistical significance of the difference between the number actually meeting 
or exceeding the standard and the number expected to meet or exceed the standard when calculated and reported is not educationally 
meaningful. 


Reporting Categories categories cannot be directly indicate strengths and weaknesses 
Table 4-5 shows the total marks compared with one another. in local educational programs. 
possible and the provincial raw Rather, the results shown in Table 
score results for the reporting 4-5 can best be used in conjunction Statistics presented in this table 
categories of the Grade 9 with parallel tables in the are based on results achieved by 
Mathematics Achievement Test. jurisdiction and school reports. 25 670 (students who wrote the 
Variations in patterns of students’ regular form and those who wrote 
It is important to stress that the responses to questions can help to Form T). 


averages on the various reporting 


Table 4-5 


Grade 9 Mathematics 
Raw Score Results by Reporting Category 
June 1992 
Maximum Raw Score 
Reporting Category Possible Score Raw Score Average | Standard Deviation 
Total Test 49* 27.7 9.7 
Major Component: 
Application and Problem Solving 31 16.5 6.6 
Knowledge and Skills : 18 11.2 3.6 
Curricular Strand: 
Numeration 13 7.2 2.8 
Ratio and Proportion 8 46 2.0 
Measurement and Geometry 11 6.0 2.6 
Data Management 7 4.3 1.9 
Algebra 10 5.5 2.4 
Taxonomy: 
Content Knowledge & Quantitative Literacy 9 5.9 2.1 
Procedural Knowledge 9 5.3 2.0 
Conceptual Understanding 20 11.4 45 i 
Problem Solving 11 5.1 2.6 | 


* Item 12 was removed from the test before scores were calculated. 
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Results for Individual 
Multiple-Choice and 
Numerical-Response 
Questions 

Table 4-6 shows the percentage of 
students who chose each alternative 


(A, B, C, and D) for each multiple- 
choice question. The correct 


response (key) for each question 

is also identified. Table 4-7 shows 
the distribution of responses for 
each numerical-response question. 
The results shown in these tables 
can best bé used conjunction with 
the parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of students’ 


responses to questions can help to 
indicate strengths and weaknesses 
in local educational programs. 


Statistics presented in these tables 
are based on results achieved by 
25 670 students (those who wrote 
the regular form and those who 
wrote Form T). 


Table 4-6 
Grade 9 Mathematics 
Results for Individual Multiple-Choice Questions 
June 1992 
Distribution of Responses (%) 
Item Component Taxonomy Curriculum Standard 
A B Cc D 
1 60.9* 5.2 13.3 20.5 Knowledge Knowledge converts a decimal expression into a 
and Skills fraction 
2 11.7 76.1* 6.5 5.6 Knowledge Knowledge converts a decimal expression into 
and Skills scientific notation with a negative exponent 
10 10.6 10.5 12.3 66.1* Knowledge Quantitative determines an approximation of the square 
and Skills Literacy root of a number 
3 15.8 30.8 3.3 50.0* | Application and Understanding applies an understanding of the subtraction 
Problem Solving of integers 
- 4 11.9 15.4 17.0 55.5* Application and Understanding express as the meaning of multiplication of 
= : Problem Solving proper fractions by a diagram 
g 5 9.3 63.6* 18.1 8.6 Application and Understanding converts a decimal into a fraction and 
Z Problem Solving notes which is the numerator and which is 
the denominator 
6 27.1 3.9 56.4* 12.4 Application and Understanding applies the laws of exponents to 
Problem Solving multiplication 
7 70.1* 5.5 2.6 21.7 Application and Understanding relates rationals and points on a number 
Problem Solving line 
8 20.8* | 75.5 3.1 0.5 Application and Problem solves a condition with a constant and 
Problem Solving | Solving a variable 
9 5.3 15.2 34.3 44,9* Application and Problem uses the divisibility rule and addition to 
Problem Solving | Solving find a numeral 
12 _ —_ _ _— _— —_— _ 
ae {| _ ff 2 
11 72.4* 47 14.6 8.1 Knowledge and Knowledge expresses three components of a ratio 
Skills in its simplest form 
§ 16 0.7 6.9 84.0* 8.3 Knowledge and Quantitative uses estimation to find an approximate 
g Skills Literacy amount of discount 
o 
ee 13 9.1 4.3 11.9 74.6* Application and Understanding recognizes the appropriate proportion for 
5 Problem Solving solving a rate problem 
fo] 
3 | 14 15.9 17.8 57.8* 8.2 Application and Understanding applies actual measurement skills and 
Problem Solving uses a scale to calculate distances 
15 7.9 49.3* 39.6 3.0 Application and Problem uses a ratio to solve a problem 
Problem Solving | Solving 


*correct answer 


Continued 
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Table 4-6 (continued) 


Distribution of Responses (%) 
Item Component Taxonomy Curriculum Standard 
A B Cc D 
17 20.7 46.8* 7.1 25.3 Knowledge Knowledge knows the characteristics of a prism 
and Skills 
22 7.7 24.5 53.4* 14.2 Knowledge Quantitative uses measuring estimation and 
and Skills Literacy quantitative literacy to establish distance 
= 18 5.4 10.3 67.5* 16.7 Knowledge Procedure recognizes the construction done by 
2 and Skills following a given procedure 
3 19 2.6 20.7 71.0% 5.6 Knowledge Procedure uses a protractor to measure an angle 
oO f 
DU and Skills 
c 
= 20 13.4 22.8 10.6 53.1* | Application and Understanding relates an understanding that the sum of 
e Problem Solving the angles of a triangle form a straight 
3 angle 
Fi 21 67.3* 46 16.2 11.8 Application and Understanding relates knowledge of a rectangular prism to 
= Problem Solving an appropriate net 
23 17.5 50.0* 18.8 13.3 Application and Problem finds the volume of a prism 
Problem Solving Solving 
24 22.6 50.1* 16.8 10.2 Knowledge Problem finds the surface area of a trench and uses 
and Skills Solving the remainder correctly 
25 18.6 8.6 56.0* 16.7 Knowledge Knowledge recalls the meaning of range and 
and Skills calculates it 
29 7.0 18.0 62.3* 12.5 Knowledge Procedure knows the meaning of median and 
and Skills calculates the number of students above it 
5 26 17.1 11.3 11.9 59.5* | Application and Understanding selects a strategy and calculates the 
E Problem Solving probability given data from a real-life 
2 situation 
= 27 9.4 63.4* 15.8 11.3 Application and Understanding identifies a representative sample of a 
& Problem Solving given population 
28 30.2 6.0 58.0* 5.7 Application and Understanding uses a population sampling technique to 
Problem Solving estimate total number 
30 11.0 13.1 14.4 61.2* | Application and Understanding uses a random sample to predict the 
| Problem Solving number needed 
31 18.0 70.0* 5.6 6.3 Knowledge Knowledge understands the meaning of like terms 
and Skills 
32 2.6 52.7* 15.8 28.7 Knowledge Procedure uses substitution to evaluate an expression 
and Skills 
£/ 33 23.2 13.8 48.1* 14.8 Knowledge Procedure solves an inequality of the form “x + a” is 
S and Skills less than or equal to “b” 
< 
34 43.7* 29.7 15.8 10.3 Knowledge Procedure solves an equation in terms of another 
and Skills variable 
35 20.9 11.9 14.7 52.2* | Application and Understanding from a table, finds a formula that shows 
Problem Solving a relationship 


*correct answer 
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Continued 


Table 4-6 (continued) 


Distribution of Responses (%) 
Item Component Taxonomy Curriculum Standard 
A B Cc D 
36 47.8* 9.1 30.1 12.8 Application and Understanding relates inequalities to an appropriate 
Problem Solving graphic representation 
37 16.6 15.1 65.9* 2.3 Application and Understanding converts an equality balance into 
Problem Solving an equation 
§ 38 41.7" | 241 17.2 16.7 Application and Understanding checks a solution for an equation involving 
2 Problem Solving integers 
39 76.0* 9.2 9.0 5.7 Application and Problem applies the meaning of average to solve 
Problem Solving Solving a practical problem 
40 8.0 14.7 55.3" 21.8 Application and Problem uses an equation or a chart to solve a 
Problem Solving Solving practical problem 


“correct answer 


Table 4-7 
Grade 9 Mathematics 
Results for Individual Numerical-Response Questions 
dune 1992 
Distribution of 
Responses (%) 
Correct Incorrect No 
Item) Response] Response |Response Strand Component Taxonomy Curriculum Standard 
1 62.0) 33.8 4.2 Numeration Knowledge Procedure applies the order of operations for 
and Skills multiplication and division using integers 
2 72.3 | 25.3 2.4 Numeration Knowledge Procedure applies the rule of exponents to solve for 
and Skills a variable 
3 45.0; 51.0 4.0 Ratio and Knowledge Procedure Calculates simple interest using a 
Proportion and Skills fractional per cent rate 
4 54.5] 42.2 3.3 Ratio and Application and Problem Solving} sets up and uses a proportional ratio to 
Proportion Problem Solving solve a practical problem 
5 21.6} 75.2 3.1 Numeration Knowledge Problem Solving; uses patterning to solve a problem 
and Skills 
6 27.2; 70.8 2.0 Ratio and Application and Problem Solving} uses ratio and proportion to solve a 
Proportion Problem Solving practical problem 
7| 485| 48.5 3.0 Measurement Application and Understanding develops a pattern for finding the sum of 
and Geometry Problem Solving the angles of a nonagon 
8 46.7/ 50.5 2.7 Measurement Application and Understanding finds the area of a regular decagon ina 
and Geometry Problem Solving practical situation 
9 479| 47.3 4.8 Measurement Application and Understanding finds the area of a parallelogram or the area 
and Geometry Problem Solving : of a triangle 
10 68.0) 29.5 2.5 Data Application and Problem Solving | uses a random sample to find defective 
Management Problem Solving ratios in a population 
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Examiner’s Observations emphases were on understanding teachers felt that they need more 


Abiievomont Test concepts and applying them in time to fully implement the 
Generally, the Grade 9 teachers context, using real-world situations _ philosophy. 

who reviewed and set standards for 04 concrete, pictorial, and : ; . 

the test felt that it was a good symbolic modes of learning. A discussion of specific areas of 
soflectianiof the Grade9 Although the program has been in strength and difficulty for Grade 9 
Mathematics program. Test operation for four years, some students follows. 

Use the information below to answer question 7. Acceptable Standard—Sample Questions and Commentary 

pa See oe Question 7 required students to recognize the scale on the number 
2 4 a 2 line and the rational numbers of the points marked. Students 


achieving the acceptable standard can do this. 
7. The rational numbers located at points Q 


and S respectively are Question 15 required students to find the amount of gold in a bracelet 
by using a ratio and then calculating the value of the gold. Students 
70.1 *A. 0.8 and *0.8 achieving the acceptable standard had difficulty solving this multi- 
5.5 B. 1.2 and 0.8 step problem. Too many were attracted to alternative C. 
2.6 C. 1.2 and +12 
217 D. ‘O4and‘04 Question 19 required students to measure an obtuse angle using the 
right-hand scale. Students achieving the acceptable standard can do 
this. 
Use the information below to answer question 15. Numerical-response question 10s required students to form a 


proportion and solve it. Students achieving the acceptable standard 
had little difficulty doing this. 


The proportion of gold in jewellery and coins is 
measured in karats (K), with 24K representing 
pure gold. 


15. The value of pure gold is $16.50/g. Ifa Overall, students achieving the acceptable standard were able to solve 
gold bracelet is marked 18K and its mass knowledge and procedural problems such as 
ets dite: Bie vale bine goin © working with number line (question 7) 
° working with scientific notation 
7.9 A. $278.20 * using simple proportion (question 10s) 
49.3 *B. $742.50 * estimating 
39.6 C. $990.00 ® measuring angles (question 19) 
3.0 D. $1237.50 * recognizing like terms and geometric constructions 


© solving exponential equations 


However, these students had difficulty 


19. The measure of <ABC is * using patterns to solve problems 


¢ knowing the meaning of terms such as range, probability, 


26 A. 37° c ‘ 

20.7 B. 82° median 

71.0 °C. 98° ° manipulating a formula with two variables 
56 D. 112° : * solving inequality conditions 


* solving multi-step problems (question 15) 


10s. A factory inspector chooses 10 radios at 
random from an assembly line. She 
tests all 10 and finds that 2 of them are 
defective. On the basis of this sample, 
about how many defective radios could 
be expected in a batch of 850? 


68.0 Solution 0170 
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9. The 3-digit number 2M3 is added to 326 
to give another 3-digit number, 5N9. If 
5N9 is divisible by 9, then Mis 


& 

> 

wo 
Somp 


= 


me & @ Pw 


Use the information below to answer questions 
23 and 24, 


The shaded region of this diagram 
represents the top view of a trench that a 
contractor dug. 


24. The contractor was required to cover the 
bottom of the trench with plastic before 
further construction could be done. If 
one roll of plastic contains 10 m?, how 
many rolls did the contractor need? 

226 A 5 

50.1*B. 86 

168 C. 52 

10.2 D. 140 

34. — Solve for nin the statement 5n - 50 = 


10¢ + 250, where nis the number of 
items bought and cis the price per item. 


43.7 *A. n=2c+60 

29.7 B. n=5e+ 200 
15.8 C. n='60 

10.3 D. n=2c+60 


A cargo plane can hold 6 trucks and 

7 jeeps, or 8 trucks and 4 jeeps. If the 
plane is loaded with jeeps only, then 
what is the maximum number of jeeps it 
would likely hold? 


27.2 Solution 0016 


Standard of Excellence-—Sample Questions and Commentary 


Question 9 required students to apply place value and the rules of 
divisibility. Students achieving the standard of excellence readily 
answered this question. | 


Question 24 required students to find the area of the shaded figure 
and consider the practical significance of the remainder in division. 
Students achieving the standard of excellence readily answered this 
question. 


Question 34 required students to solve a condition for a variable in 
terms of another variable and a constant. Most students achieving the 
standard of excellence correctly answered this question; however, 
some selected alternative B. 


Numerical-response question 6s required students to use a ratio to 
solve a practical problem. Students achieving the standard of 
excellence had difficulty solving this question. 


Students who achieved the standard of excellence demonstrated more 
success in solving multi-step problems than other students did. 
Specifically, students achieving the standard of excellence could 


* solve practical problems (question 24) 

* solve place value problems involving divisibility rules (question 9) 
* solve and check conditions (question 34) 

® use patterns to solve problems 


However, these students had difficulty 


° solving problems with two variables 
* solving problems involving computation with ratios (question 6s) 
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Performance-Based 
Assessment 
General Description 


In addition to answering the 
multiple-choice and numerical- 
response questions on the 
achievement test, 504 randomly 
selected students from 22 schools 
throughout the province 
participated in performance-based 
assessment. 


Performance-based assessment 
was developed to assess student’s 
higher order thinking skills in real- 
life problem-solving situations. 

The tasks assessed aspects of 
mathematics that could not be 
measured adequately by paper- 


Table 4-8 


and-pencil tests in which only the 
answer is recorded and marked. 


The six activities that were used 
are briefly described in Table 4-8. 
They engaged students in tasks 
that allowed for a variety of 
strategies to be used, the use of 
manipulatives, and the collection 
of information. Manipulatives 
were available for direct use in 
solving the problems and for 
collecting information. These 
performance-based assessment 
tasks provided information about 
integrated learning across and 
within subject areas. They 
involved the active participation of 
the students in new and different 
situations. 


Students were provided a writing 
experience in mathematics when 
asked to explain the strategies 
they used and how these strategies 
were carried out to solve the 
problems. Their responses provide 
another picture of what students 
know and are capable of doing in 
mathematics. 


A group of experienced Grade 9 
Mathematics teachers met in July 
1992 and established scoring 
standards. All student responses 
were scored following the standards 
set, with about 20% of papers being 
rescored to ensure marker 
consistency. The results are shown 
in Table 4-9 and Figure 4-3. 


Grade 9 Mathematics 
Performance-Based Assessment Activities 
Activity Name Content Area Learning Expectation/Skill Area 
1 Seating Place Numeration Analysis of Problems 
Measurement Use of a Model 
Use of Factors 
Relation between Perimeter and Area 
2 Highway to Measurement Application of Measurement 
Mathematics Ratio and proportion Use of Proportion 
Use of Map Legend 
3 Solar Cylinder Geometry Application 
Measurement Analysis 
Operations Synthesis—Subproblems 
Use of Measuring Skills 
Finding Area 
Checking Reasonableness of Answer 
4 Ski Jumping Data Collection Data Collection and Recording 
Data Management Display of Data 
Numeration Analysis of Data 
Interpretation of Data 
5 Bucket of Beans Measurement Analysis 
Estimation 
Checking Reasonableness of Answer 
6 Packaging Numeration Analysis 
Monitoring 
Use of Factors 
Finding Greatest Common Factor 
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Table 4-9 N=503 
Grade 9 Mathematics 
Performance-Based Assessment Results 


Activity 1 Activity 2 Activity 3 Activity 4 Activity 5 Activity 6 All 
Student Achievement (%) (%) (%) (%) (%) (%) Activities (%) 
Beyond Grade 9 8.3 15.1 9.1 2.4 15.9 50.1 16.8 
E 2 At Grade 9 41.7 32.0 13.5 36.4 47.7 12.1 30.6 
2 > Not Yet at Grade 9 34.4 44.1 30.4 48.3 27.8 18.5 33.9 
&@ | Totally Misunderstood or 15.5 8.7 46.9 12.9 8.5 19.3 18.6 
Left Blank 
8 Beyond Grade 9 6.0 17.7 11.5 4.0 23.2 32.6 15.8 
Hy At Grade 9 35.6 38.6 17.3 40.4 40.4 39.8 35.1 
s Not Yet at Grade 9 41.0 38.8 35.8 44.9 30.0 21.1 35.3 
£ Unclear and 17.5 6.8 35.4 10.7 6.4 6.6 13.9 
E Inappropriate 
° 
Figure 4-3 
Grade 9 Mathematics 
Percentage of Students Achieving At or Beyond Grade 9 Expectation 
100 
80 
60 - 
x 
40 
20 
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Pos L a oes i? SOR % i, TRE a5 
Activity 2 Activity 3 Activity 5 Activity 6 All Activites 


Problem Solving 


=| Communication 


Examiner’s Observations The descriptive-analytic ¢ students were more successful in 
Student responses to each activity assessment showed that solving problems when 
were assessed using two types of manipulatives could be used 
scoring methods. Descriptive- * about half of the students were directly to solve the problem 
analytic scoring identified the not able to solve open-ended fe 
strategies that students used to problems successfully Results of the holistic scoring on 
complete the tasks and the each of the tasks showed that 
accuracy of their responses. A * successful students used a wide 
holistic scoring guide was used for range of strategies in problem * students who were able to apply 
rating students’ problem-solving solving the problem-solving model were 
and communication skills. more successful in solving 

* students in general preferred to problems 


solve problems using the 
concrete mode rather than the 
symbolic mode 
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* students who were able to 
apply the problem-solving model 
could also clearly communicate 
the process and results in 
writing 


there is no gender difference in 
students’ ability to solve 
problems; however, girls were 
better than boys in 
communicating 


A more detailed description of the 
assessment tasks and scoring 
criteria, and samples of students’ 
performances are provided in the 
booklet Samples of Students’ 
Responses from the June 1992 
Grade 9 Mathematics Performance- 
Based Assessment Tasks. 


SS ss a 


Relationship between 
Performance-Based 
Assessment and the 
Achievement Test 


Results show a positive correlation 
between achievement test scores 
and scores on the performance- 
based assessment. The highest 
correlation occurs between the 
problem-solving component of the 
achievement test and performance- 
based assessment. 


Both performance-based 
assessment and achievement test 
results indicate that Grade 9 
students have weak problem- 
solving skills in mathematics. 


SS is 


Contexts for 
Learning Study 
General Description 


In June 1992, 57 Grade 9 
Mathematics teachers and 450 
students from across the province 
participated in a pilot study 
designed to identify and examine 
relationships among various 
contexts for learning and their 
effect on student achievement. 
Students responded to questions 
related to the attitudes outlined 
in the Program of Studies for 
Mathematics. Teachers reported on 
the types and frequency of use of 
instructional strategies, activities, 
classroom resources, and 
manipulatives. Results of this 
study are reported in Table 4-10 
and Table 4-11. 


SSS esha ca as 


Table 4-10 
Grade 9 Mathematics 
Contexts for Learning: Percentage Distribution of Student Responses 
Oe No Strongly Strongly 
Statement ‘response disagree Disagree Undecided Agree agree 
1. There are usually different ways to solve 0 0.4 0.2 2.2 53.1 44.0 
math problems. 
2. Knowing how to solve a math problem is as 2.0 1.3 8.9 5.6 37.7 46.2 
important as getting the right answer. ; 
3. | Mathematics is useful for solving every- 0 0.7 4.7 13.1 48.2 33.1 
day problems. 
4. | Mathematics helps me with: 
a. taking care of my money. 5.0 0.9 2.2 49 474 44.3 
b. preparing for high school. 15.0 0.5 3.4 12.9 47.1 36.1 
c. balancing my time commitment for 
different activities. 19.0 3.0 16.0 29.7 35.7 15.5 
d. getting a job. 12.0 1.4 7.3 13.9 39.5 37.7 
5. | am good at mathematics. 7.0 5.0 14.7 23.5 46.5 10.4 
6. When |! leave school, | will not need most of 
the math | have leamed. 6.0 27.5 33.1 19.1 15.3 5.0 
7. Being good at mathematics is important in 
getting a good job. 5.0 0.4 3.6 14.4 45.4 36.2 
8. | like math puzzles. 5.0 9.9 18.7 27.2 32.4 11.9 
9. | like to be challenged in math. 5.0 7.6 19.3 23.8 34.8 14.4 
Continued 
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Table 4-10 (continued) 


No Strongly Strongly 
Statement response disagree Disagree Undecided Agree agree 
10. | feel successful in math. 5.0 7.0 16.0 27.9 38.0 11.2 
11. The more | learn in math, the more interesting 
it becomes. 4.0 5.6 12.1 26.5 36.8 19.1 
No 
Statement response Never Rarely Sometimes Often Always 
12. | make sure | try every problem, even if | 
cannot solve them all. 8.0 11 5.9 27.6 39.8 25.6 
13. When | do not understand a problem in 
mathematics, |: 
a. try different ways to solve it on my own. 19.0 2.1 11.1 32.3 38.3 16.2 
b. ask another student for help. 18.0 0.7 10.9 38.0 40.3 10.2 
c. wait and ask someone at home. 24.0 16.0 39.0 28.9 14.6 1.6 
d. give up. 22.0 25.9 41.8 25.0 6.3 0.9 
14. If |had a complicated math puzzle or problem 
that | couldn’t solve in 10 minutes, the next 
thing | would likely do is: 
a. continue trying until | solved it. 22.0 47 23.8 37.1 26.2 8.2 
b. set a time limit for trying. 23.0 19.0 27.9 32.1 18.5 2.6 
c. give up. 34.0 21.6 40.4 28.1 8.2 1.7 
d. get help to solve the puzzle. 26.0 0.7 6.4 33.0 48.1 11.8 
e. get a clue to help and continue trying. 28.0 1.4 11.1 29.4 47.4 10.7 
15. If, after solving a particularly challenging math 
puzzle, | compared my solution with a 
classmate and found that | had a different 
answer, | would: 
a. assume my own solution is incorrect. 20.0 7.7 28.6 47.7 12.3 3.7 
b. share my solution with others to see if they 
also have solutions that work. 22.0 2.8 6.5 32.9 46.7 11.0 
c. assume that other classmates may also have 
different solutions, but that mine is still good. 26.0 5.0 14.4 43.6 30.7 6.4 
d. see if | can solve the puzzle another way so 
that | would have two solutions. 19.0 1.1 19.0 35.0 28.1 6.7 
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Table 4-11 
Grade 9 Mathematics 


Contexts for Learning: Percentage Distribution of Teacher Responses 
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No 
Statement response Never Rarely Sometimes | Often Always 
1. | encouraged students to try different 
strategies in solving problems this year. 2.0 _ — 11.3 60.4 28.3 
2. This year | demonstrated when | was solving 
a problem that | didn’t always know how to 
solve it immediately. 3.0 1.9 9.6 40.4 36.5 11.5 
3. This year | encouraged students to share 
responses and consider the merits of each. 1.0 — 3.7 27.8 51.9 16.7 
4. This year | encouraged students to bring in 
problems from home or make up their own. 1.0 13.0 40.7 35.2 9.3 1.9 
5. | was enthusiastic about problem solving 
this year. 1.0 _ 1.9 16.7 51.9 29.6 
6. This year | emphasized willingness and 
perseverance rather than speed. 1.0 _ 3.7 20.4 55.6 20.4 
7. encouraged a variety of solutions 
this year. 1.0 _— _— 22.2 46.3 31.5 
8. | provided students with problems at the 
appropriate difficulty level. 1.0 _ _ 13.0 68.5 18.5 
9. This year | personalized problems whenever 
possible. 1.0 _— 7.4 33.3 50.0 9.3 
10. Most of my students liked to be challenged 
in math this year. 3.0 _ 21.2 51.9 25.0 1.9 
11. This year when my students did questions 
involving new material, most of them: 
a. set their own time limit for working on the 
questions. 2.0 5.7 28.3 34.0 32.1 _ 
b. waited until they had my assistance 
to begin. 1.0 5.6 24.1 48.1 22.2 _ 
c. discussed the new material with a 
classmate. 2.0 — 3.8 26.4 58.5 11.3 
d. waited until the questions were answered 
by someone else. 2.0 _ 40.4 46.2 13.5 _ 
12. This year when my students were presented 
with a “novel problem,” most of them appeared: 
a. motivated. 3.0 _ 7.7 51.9 40.4 ot 
b. frustrated. 3.0 3.8 11.5 65.4 19.2 _ 
c. bored. 6.0 41 38.8 46.9 10.2 = 
13. | used the following with my students this year: 
a. Base ten blocks 5.0 78.0 18.0 4.0 —_ = 
b. Place value chart and markers 3.0 44.2 15.4 36.5 1.9 1.9 
c. Number line 1.0 — 1.9 37.0 50.0 11.1 
d. Fraction bars or fraction kits 3.0 50.0 17.3 17.3 15.4 — 
Continued 


Table 4-11 (continued) 


No 
Statement response Never Rarely Sometimes Often Always 
e. Calculators 3.0 _ 1.9 5.8 42.3 50.0 
f. Computers 4.0 54.9 25.5 15.7 3.9 — 
g. Decimal Squares 7.0 79.2 10.4 10.4 _ _ 
h. Shapes (cylinders, cones, cubes, etc.) 2.0 1.9 1.9 35.8 41.5 18.9 
i. Geometry Set 1.0 1.9 1.9 31.5 46.3 18.5 
j. Mira 4.0 80.4 9.8 5.9 2.0 2.0 
k. Geoboards 3.0 76.9 15.4 7.7 _ _— 
|. Balance Scales 2.0 58.5 15.1 18.9 5.7 1.9 
m.Aigebra Tiles 3.0 69.2 15.4 9.6 1.9 3.8 
14. How often did working with manipulative 
materials help your students understand 
math this year? 1.0 1.9 9.3 61.1 25.9 1.9 
15. This year when my students needed or wanted 
manipulatives to help them, they: 
a. helped themselves to what they needed. 2.0 20.8 24.5 35.8 13.2 5.7 
b. asked me for permission to get what 
they needed. 16.0 26.0 40.0 14.0 4.0 
16. When | taught the problem-solving component 
of my math program this year: 
a. [taught problem-solving strategies as a 
separate unit and integrated problem 
solving into the math program throughout 
the year. 15.1 15.1 18.9 34.0 17.0 
b. | taught problem solving as a separate unit. 54.2 16.7 10.4 12.5 6.3 
c. lintegrated problem-solving into the math 
program throughout the year and taught 
problem-solving strategies as they were 
needed. _ 2.1 4.3 57.4 36.2 
17. When dealing with solutions to problems in 
mathematics this year: 
a. students reported their results to the 
whole class. 1.9 13.2 64.2 17.0 3.8 
b. students discussed their solutions with 
each other. _ 2.0 49.0 39.2 9.8 
c. students wrote up their solutions and 
handed them in. 3.8 9.6 61.5 19.2 5.8 
Less than Several 
No once a Once a times a 
Statement Response Never week week week Every day 
18. This year my students did the following in 
math class: 
a. Used a computer 1.0 70.4 27.8 _ 1.9 _ 
b. Used a calculator 1.0 _ 1.9 = 42.6 55.6 
c. Used manipulative materials 3.0 5.8 51.9 30.8 11.5 _ 
d. Wrote in a journal 1.0 81.5 9.3 1.9 3.7 3.7 
e. Did exercises individually 2.0 1.9 3.8 5.7 67.9 20.8 
f. Listened to a lesson presented to the 
whole class 3.0 1.9 7.7 3.8 65.4 21.2 
Continued 
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Table 4-11 (continued) 


Less than Several 
No once a Once a times a 
Statement Response Never week week week Every day 
g. Worked in small groups 1.0 5.6 16.7 27.8 44.4 5.6 
h. Wrote a math test or quiz 1.0 1.9 40.7 46.3 7.4 3.7 
i. Solved problems in small groups 1.0 3.7 37.0 24.1 29.6 5.6 
j. Independently worked on problem solving 1.0 3.7 20.4 35.2 37.0 3.7 
k. Corrected homework or practice questions 
with the whole class 1.0 — 13.0 3.7 54.4 25.9 
19. This year | used math manipulatives to assist 
my demonstration/instruction 4.0 _ 56.9 23.5 19.6 _ 
20. | conferenced with individual students 2.0 1.9 18.9 11.3 35.8 32.1 


Examiner’s Observations 


Student Attitude 
Questionnaire—Contexts for 
Learning 

The student questionnaires reveal 
that students generally feel that: 


© knowing how to solve a math 
problem is as important as 


stronger tendency to be open to 
multiple and different solutions to 
problems. 


Of the students achieving in the 
acceptable standard but not the 
standard of excellence, 66% feel 
that they are good in mathematics 
but only 55.1% indicated that they 


challenges in mathematics. While 
fewer students in this group 
indicated they would persist in 
trying to solve a difficult problem 
on their own, over half reported 
they would get help from another 
student. 


Teacher Questionnaire— 


getting the right answer were successful in mathematics. Contexts for Learning 
Although this group of students The teacher questionnaires reveal 
i iathemiatinstisvadehalan indicated positive attitudes about that teachers generally 


everyday life and necessary for 
getting a good job 


© they can persist but confidence is 
rather low 


Relationship between 
Student Attitudes and the 
Achievement Test 

Students achieving the standard 
of excellence or higher on the 
achievement test were more likely 
to report positive attitudes about 
mathematics. They reported that 
mathematics was interesting to 
study and that they enjoyed 
puzzles and challenges in 
mathematics. This group of 
students were more likely to be 
persistent and confident when 
trying to solve difficult or lengthy 
problems. They also showed a 
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mathematics, they were less likely 
than the group achieving the 
standard of excellence or higher to 
enjoy puzzles and challenges in 
mathematics. A larger proportion 
of this group of students reported a 
reliance on group work to assist 
them in solving problems and 
validating solutions than those 
achieving below or above them. 


Students achieving below the 
acceptable standard or higher 
indicated they believed that 
mathematics was important, but 
only 22.2% felt they were good at 
mathematics and only 18.1% 
expressed a belief that they are 
successful in mathematics. This 
group of students was less likely to 
find mathematics interesting and 
felt negatively toward puzzles and 


® encourage a variety of problem- 
solving strategies 


* integrate problem solving 
through the strands 


¢ are enthusiastic in their 
approach toward problem solving 


¢ do not use manipulatives to a 
great extent 


Relationship between Contexts 
for Learning and the 
Achievement Test 

In this discussion, classrooms 
where most students met or 
exceeded the acceptable standard 
are referred to as high achieving 
classrooms, and those classrooms 
where few students met or 


exceeded the acceptable standard 
are referred to as low achieving 
classrooms. Of the high achieving 
classrooms, 58% use manipulative 
materials in math class one or 
more times a week, whereas 24% 
of low achieving classrooms use 
manipulative materials with the 
same frequency. Students in 74% 
of high achieving classrooms 

help themselves to whatever 
manipulatives they need or want 
to help them understand math, 
whereas 19% of low achieving 
classrooms are offered the same 
opportunity. Teachers indicate that 
in 53% of high achieving 
classrooms and 33% of low 
achieving classrooms, computers 
are used often. Calculators are 
used every day in 63% of high 
achieving classrooms and 53% of 
low achieving classrooms. 


Of the classrooms surveyed, 93% 
have problem solving integrated 
into the mathematics program 
throughout the year, with problem- 
solving strategies taught as 
needed. Differences between high 
achieving and low achieving 
classes can be seen in the way 
students respond to new material 
and in how achievement in 
mathematics is assessed. When 
students do questions involving 
new material, students in 35% of 
low achieving classrooms indicated 
that they often wait until they 
‘have the teacher’s assistance to 
begin, whereas only 10% of 
students in high achieving 
classrooms wait for the teacher’s 
assistance. In 77% of high 
achieving classrooms and 52% of 
low achieving classrooms, students 
discussed new material with a 
classmate before doing the 
questions. When problem solving, 
students in 22% of high achieving 
classrooms frequently report their 
results to the whole class, whereas 


students in 6% of low achieving 
classsrooms dealt with results in a 
similar manner. Students in 68% 
of high achieving classrooms write 
a math test or quiz one or more 
times a week, whereas 47% of low 
achieving classrooms write a test 
or quiz with the same frequency. 


The contexts for learning show 
that differences do exist in how 
educational programs are 
implemented. 


Issues 


Both performance-based 
assessment and achievement test 
results indicate that Grade 9 
students have weak problem- 
solving skills in mathematics. 
Performance-based assessment 
shows that students in general 
prefer to solve problems using the 
concrete mode rather than the 
symbolic mode and are more 
successful in solving problems 
when manipulative materials can 
be used directly to solve the 
problem. High achieving 
classrooms used a greater variety 
of manipulative materials more 
frequently than low achieving 
classrooms. 


These findings, along with the 
focus placed on use of manipulatives 
in developing problem-solving 
skills by the new Program of 
Studies in 1988, suggest a direction 
for improving instruction in 
mathematics. 
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Section 5 
! 


Achievement by Gender 


VY 


Information on the gender of 
students who wrote the provincial 
achievement tests has been 
collected and data have been 
analyzed and reported since 1989. 


This section of the report answers 
the following questions: 


¢ What is the proportion of males 
and females who wrote the 1992 
achievement tests? 


* Is the percentage of males and 
females meeting standards the 
same in each subject? 


* Are the 1992 results for 
individual subjects similar to or 
different from those of 1989 
through 1991? 


Observations and 
Discussion 


What is the proportion of males 
and females who wrote the 1992 
achievement tests? 

Results for 1992, presented in 
Figure 5-1, reveal that more males 
than females wrote the 
achievement tests at each grade 
level. This pattern is similar to 
the data from 1989 through 1991, 
with the exception of 1990 when 
more females than males wrote 
the Grade 9 English Language 
Arts test. 


Figure 5-1 
Number of Achievement Tests Written by Gender* 
June 1992 
thousands 
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18 b 9% ~ 49.0% . 50.8% .. 49.1% pean aati orton 
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Social Studies English Language Arts Mathematics 
(Gr. 3) (Gr. 6) (Gr. 9) 
N= 31 633 N= 33 685 N= 27 855 


Male Y Female 


* Figures do not include tests where students did not identify their gender. The number of 
students who did not provide gender data is as follows: 
Grade 3 Social Studies—21 students 
Grade 6 English Language Arts—32 students 
Grade 9 Mathematics—33 students 
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Is the percentage of males and 
females meeting standards the 
same in each subject? 

Results for 1992 presented in 
Figures 5-2 and 5-3 reveal that 
females outperformed males in 
Grade 3 Social Studies and Grade 6 
English Language Arts. Results for 
1992 Grade 9 Mathematics show 
that males outperformed the 
females. 


Are the 1992 results for individual 
subjects similar to or different from 
those of 1989 through 1991? 

The pattern for the 1992 Grade 3 
Social Studies results is similar to 
the 1989 Grade 6 Social Studies 
result: females outperformed males, 
particularly at the standard of 
excellence. In 1991, however, 
males marginally outperformed 
females in Grade 9 Social Studies. 


The pattern of females 
outperforming males for the 
Grade 6 English Language Arts is 
similar to both the 1989 Grade 3 
and the 1990 Grade 9 English 
Language Arts results. 


The 1990 Grade 3 Mathematics 
results and the 1991 Grade 6 
Mathematics results show a 
similar pattern to the 1992 Grade 9 
results of males outperforming 
females. However, the difference in 
achievement between males and 
females is greater at the Grade 9 
level. 
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Figure 5-2 

Percentage of Students Achieving Acceptable Standard or 
Higher on the Total Test by Gender 

June 1992 


100 


B27 % = SAA a op a ae 


60 -4- $e BT pee pees Maes 
% 
40 +1. FE Ff oF prc ED FF err eeereeeeeens te et ee eeeeeey 


20 Cen Sia i See Soa Ci a Cree SSeS Me se ae eee 


Social Studies English Language Arts Mathematics 
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* 85% of students were expected to achieve the acceptable standard or higher on the total 
test. 


Figure 5-3 

Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test by Gender 

June 1992 
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Social Studies English Language Arts Mathematics 
(Gr. 3) (Gr. 6) (Gr. 9) 


Male Female 
ZL 


*15% of students were expected to achieve the standard of excellence or higher on the total 
test. 


Section.6 


t 


Achievement by Age 


Information on the age of 

students who wrote the provincial 
achievement tests was first 
collected in June 1990. The 1990 
and 1991 data were analyzed and 
reported in the 1990 Achievement 
Testing Program Provincial Report 
and again in the 1991 Achievement 
Testing Program Provincial 


Report. This section of the report 
answers the following questions: 


¢ What is the age distribution of 
students who wrote the 1992 
achievement tests? 


¢ What relationship, if any, does 
age appear to have with 


achievement as measured by 
the 1992 provincial achievement 
tests? 


Age Groups 


This year, for the first time, 
information concerning both the 
year and the month of birth at the 
time of testing was collected. 


Table 6-1 
Number of Achievement Tests Written by Age 
June 1992 
Age 
(6 month intervals) Grade 3 Social Studies Grade 6 Language Arts Grade 9 Mathematics 
Number Percentage Number Percentage Number Percentage 
Under 7 51 0.2 
7 (84-89 mos.) 18 0.1 
7.5 (90-95 mos.) 272 1 0.0 
8 (96-101 mos.) 1348 
10 (120-125 mos.) 1678 5.3 0.0 
10.5 (126-131 mos.) 198 0.6 
11 (13; 110 0.3 0.0 
1501 27 0.1 
18 0.1 
12 4 0.0 
13 : 8 0.0 
13.5 (162-167 mos.) 3 0.0 
14 (168-173 mos.) 7 0.0 
16 (192-197 mos.) 1538 5.5 
16.5(198-203 mos.) 305 14 
17 or older 121 0.4 
unknown 362 1.4 184 0.5 147 0.5 
Total 31654 100.0 33717 100.0 27888 100.0 


Age on the fest day (June 9, 1992) 


An empty area indicates that there were no students 
The percentages for Grade 6 do not add up to 100.0 because of rounding. 
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EES 


We were therefore able to * or 
determine the age of students with Figure 6-1 
greater precision than in previous Percentage of Students Achieving Acceptable Standard or 
studies. Each student was Higher on the Total Test by Age 

categorized as being in the first or June 1992 

the last half of his or her age year 

(e.g., 7 years = 84 to 89 months Grade 3 Social Studies 

7.5 years = 90 to 95 months). Table 
6-1 shows the exact age range in 
months of each group. Figure 6-1 
and Figure 6-2 show the proportion 
of students in each age group who 
met standards. 


<7yrs 7yrs 8 yrs 8 yrs 9 yrs 9 yrs 10 yrs 11 plus 
6 mos 6 mos 


Age of Students on June 9, 1992 
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Figure 6-2 
Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test by Age 
June 1992 
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Observations 
and Discussion 


Grade 3 Age Groups 

For the year 1992, 40.5% of tested 
Grade 3 students were 8 years old 
at the time of testing and 50.8% 
were 9 years old. Of the tested 
students, 1.2 % were younger than 
8 years old on June 9 and 6.4% 
were older than 9. It is interesting 
to note that there were 40 students 
aged 12, 13, and 14 in Grade 3. It 
may be that a broader group of 
special needs students wrote the 
Grade 3 test. 


Only 51 age 6 students are 
reported to have written the Grade 3 
test and 290 age 7 students wrote. 
Proportionately fewer of the 
students aged 7 and younger were 
able to meet the standards. This is 
also true of students aged 9.5 and 
older. 


Grade 6 Age Groups 

For the year 1992, 41.2% of tested 
Grade 6 students were 11 years old 
at the time of testing and 51.1% 
were 12 years old. Of the tested 
students, 0.3 % were younger than 
11 years old on June 9 and 6.6% 
were older than 12. While there 
were some students aged 9, 15, 
and 16, the number was very 
small. There seemed to be a 
slightly narrower distribution of 
students in Grade 6 than in 

Grade 3. 


Of the 6,206 students aged 12.5 
and older, proportionately fewer 
were able to meet the acceptable 
standard and the standard of 
excellence. The difference is quite 
large. A larger proportion of the 
older students are not doing as 
well as might be expected. 


Grade 9 Age Groups 


For the year 1992, 42.1% of tested 
Grade 9 students were 14 years old 
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at the time of testing and 50.1% 
were 15 years old. Of the tested 
students, 0.2 % were younger than 
14 years old on June 9 and 7.0% 
were older than 15. There were 
121 students who wrote the Grade 9 
test who were 17 years of age or 
older. 


For Grade 9, there appears to be a 
negative relationship between age 
and student achievement. Younger 
students generally achieve higher 
that do older students, with regard 
to both the acceptable standard 
and the standard of excellence. 


Summary 


In 1990 and 1991, a pattern of 
younger students outperforming 
older students was noticed in each 
of the three grades tested. In 1992, 
the collection of both month and 
year of birth to determine age 
permitted grouping of students 
within narrower age ranges. The 
data from these groupings seem to 
indicate that this pattern does not 
hold true for very young students 
in grades 3 and 6. In these grades, 
students in general who are 
younger than the mainstream of 
students did not necessarily 
perform better. 


In general, it would appear that 
students who are older than the 
mainstream did not achieve as 
well as expected. This lower 
achievement of older students in 
the grade levels tested may be 
attributed to the interaction of 
several factors. The most likely 
factors are repeating students, 
late entrants into formal schooling, 
and transferees from other 
provinces or countries where there 
may be curricular or language 
differences. 


Schools and jurisdictions 

whose results reflect negative 
relationships between age and 
achievement are encouraged to 
identify the specific factors 
operating within their own 
classrooms and schools. Accurate 
identification of such factors would 
be the first step in the development 
of plans to help older students 
achieve to their full potential. 


Student Evaluation Branch 
welcomes observations from school 
administrators on the factors they 
identify as being major contributors 
to the prevailing negative 
relationships between age and 
achievement. 
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Section 7 


Achievement by 
Grade Level 


y 


This section is an exploratory 
approach to reporting results from 
the 1992 achievement tests. It 
attempts to provide answers to the 
following question: 


What percentage of grades 3, 
6, and 9 Alberta students 
® may be achieving beyond 
grade level 
* are achieving at grade level 
® may be achieving at grade level 
* are not yet achieving at grade 
level 
as measured by the June 1992 
achievement tests? 


To facilitate implementation of 
a results-based curriculum, this 
section analyzes and reports 
student achievement by grade 
“levels”. 


Within each curriculum, specific 
outcomes, expressed as the 
knowledge and skills to be 
acquired and developed, are 
arranged into sequences reflecting 
the developmental nature of 
learning and expectations for a 
learner’s increasing competence 
or confidence in the subject area. 


The 1992 achievement tests were 
designed to measure student 
achievement within the given 
grade level. However, a number of 
questions on the achievement tests 
could be considered to measure 
more advanced levels of 
performance. For reporting 
purposes only, we have classified 
students within the grade levels 
framework based on the following 
four definitions: 


. May Be Achieving Beyond 


Grade Level 

Students who may be achieving 
beyond grade level are those 
students who achieved the 
standard of excellence for both 
major components of the test. 


. Achieving At Grade Level 


Students who are achieving at 
grade level are those students 
who achieved the acceptable 
standard on the total test and 
those students who achieved the 
standard of excellence on the 
total test but not on both 
components. 


May Be Achieving At Grade 
Level 

Students who may be achieving 
at grade level are those 
students who met the 
acceptable standard on one of 
the major components of the 
test but not on the total 

test. 


. Not Yet Achieving At Grade 


Level 

Students who are not yet 
achieving at grade level are 
those students who did not meet 
the acceptable standard on 
either major component of the 
test. 


Results from the 1992 
achievement tests showing 
distributions of grade level 
achievement are presented 

in Figure 7-1 and Tables 7-1 

to 7-3. This experimental 
analysis shows the wide range 
of student achievement within a 
nominal grade level. 
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Figure 7-1 
Distribution of Grade Level Achievement 
June 1992 
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Table 7-] 
Grade 3 Social Studies 

Distribution by Grade Level Achievement 
dune 1992 


Grade Level 3 Number Per Cent 
Students Who May Be Achieving Beyond Grade Level 3 898 12.3 
Students Who Are Achieving At Grade Level 22 533 71.2 
Students Who May Be Achieving At Grade Level 1 768 5.6 
Students Who Are Not Yet Achieving At Grade Level 3 455 10.9 
Total 31 654 100.0 
Table 7-2 
Grade 6 English Language Arts 
Distribution by Grade Level Achievement 
June 1992 
Grade Level 6 Number Per Cent 
Students Who May Be Achieving Beyond Grade Level 1 798 5.3 
Students Who Are Achieving At Grade Level 23 698 70.3 
Students Who May Be Achieving At Grade Level 4335 12.9 
Students Who Are Not Yet Achieving At Grade Level 3 886 11.5 
Total 7 33 717 100.0 
Table 7-3 
Grade 9 Mathematics 
Distribution by Grade Level Achievement 
June 1992 
Grade Level 9 Number Per Cent 
Students Who May Be Achieving Beyond Grade Level 1 782 6.4 
Students Who Are Achieving At Grade Level 17 025 61.0 
Students Who May Be Achieving At Grade Level 3 420 12.3 
Students Who Are Not Yet Achieving At Grade Level 5 661 20.3 
Total 27 888 100.0 


Section 8 


Achievement Over Time 


An important task of Alberta 
Education is to measure and report 
changes in student achievement 
over time. Since 1983, the Student 
Evaluation Branch has been 
measuring the achievement of 
students in grades 3, 6, and 9 in 
the subject areas of language arts, 
mathematics, science, and social 
studies. The administration of the 
Achievement Testing Program 
follows a four-year cycle for each 
subject within each grade level. 

In 1984, 1988, and 1992, Social 
Studies was tested in Grade 3, 
Language Arts was tested in Grade 
6, and Mathematics was tested in 
Grade 9. 


This section of the report answers 
the following question: 


¢ Has achievement, as measured 
by individual provincial 
achievement tests in Grade 3 
Social Studies, Grade 6 Language 
Arts, and Grade 9 Mathematics, 
changed since 1984 and 1988? 


Three studies were carried out. 
The first was designed to compare 
achievement since 1984 on the 
multiple-choice questions in Grade 3 
Social Studies, Grade 6 Language 
Arts, and Grade 9 Social Studies. 


The second study was designed to 
detect practice effects that are 

the result of students in the 
achievement-over-time groups 
using multiple-choice questions 
from the 1988 and/or 1984 tests as 
practice questions. The presence 


of practice effects may alter 
conclusions about achievement- 
over-time results for the multiple- 
choice sections. 


The third study was designed to 
compare achievement since 1984 
and 1988 on the written-response 
part of the Grade 6 Language Arts 
test. The results of these special 
studies follow. 


Multiple-Choice Study 
Design 

Before 1991, achievement-over- 
time studies carried out by the 
Student Evaluation Branch 
involved the re-administration 
of earlier tests to a sample of 
students who were also writing 
the current tests. Because the 


‘same students had to write two 


tests, the re-administration was 
either one week before or one 

week after the current test 
administration. Subsequently, 
motivational levels and state of 
preparation were likely different 
from when the tests were written 
during the original administrations 
and the results therefore less valid. 
The additional testing time 
required of students participating 
in the study was a concern. 


To address these issues a new 
design, first implemented in 1991, 
was used. For each subject, a 
random group of students was 
chosen as the achievement-over- 
time (AOT) group. The AOT group 
was randomly divided into three 


sub-groups. One sub-group wrote 
the current 1992 test so that we 
could verify that the AOT group 
was representative of the province. 
Each of the other two sub-groups 
was assigned to write either the 
1988 AOT test or the 1984 AOT 
test. 


The AOT tests were developed to 
match the blueprints of the 1992 
test. Furthermore, these tests 
were administered at the same 
time and under the same conditions 
as the regular achievement tests in 
the rest of the province, as a means 
of maintaining the validity of the 
testing for the AOT students. 


Each AOT test consisted of roughly 
equal portions of questions from 
the year being compared (1988 or 
1984) and questions from the 
current 1992 test. The 1992 
portions of the various tests allow 
us an additional verification that 
the random sub-groups in the 
study are representative of the 
province. In addition, the current 
1992 portion allows the calculation 
of a 1992 test score for each of the 
students in the AOT sample. The 
1988 portion of the AOT test is 
used to compare 1992 student 
performance with student 
performance in 1988. The 1984 
portion allows comparison with 
the 1984 student population. 
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Table 8-1 shows the composition of 
the achievement-over-time (AOT) 
tests in terms of the number of 
questions they contained from the 
1984, 1988, and 1992 tests. 


Table 8-1 
Comparison of the Achievement-Over-Time Tests by the Number of Questions from the Previous 
and Current Tests 

1992 1988 1984 

AOT Test AOT Test AOT Test 

Grade 3 Social Studies 
Number of 1984 Questions 0 0 25 
Number of 1988 Questions 0 26 ¢) 
Number of 1992 Questions 50 24 25 
Grade 6 Language Arts 
Number of 1984 Questions 0 0 20 
Number of 1988 Questions 0 24 0 
Number of 1992 Questions 50 26 30 
Grade 9 Mathematics 
Number of 1984 Questions 0 0 16 
Number of 1988 Questions 0 16 0 
Number of 1992 Questions 49** 31* 30** 


* The numerical-response section contain three questions and four that are respectively not on the 1992 test or on the AOT 88 or AOT 84 
tests. 
** One question on the Grade 9 1992 AOT test was dropped. This question did not appear on either the 1988 AOT or 1984 AOT forms. 


Methodology 


Sampling of the schools involved 
in the study was carried out 
independently in grades 3, 6, and 
9. Because of project design 
considerations, only schools with 
24 or more students in regular 
English programs who wrote 
achievement tests in 1991 in the 
grade being sampled were included 
in the groups from which samples 
were drawn. At each of the three 
grade levels, all eligible schools 
were identified and listed. From 
each of these lists, random samples 
of schools were selected. The 
sample sizes were targeted for 
approximately 3 000 students at 
each grade level. Since the 
sampling was carried out 
independently at each grade level, 
some schools were randomly 


&0 


selected to participate at more 
than one grade. 


Before test materials were shipped, 
the participating schools submitted 
copies of their class lists to the 
Student Evaluation Branch. 
Within each class, students were 
randomly assigned to write either 
the AOT 1984 test, the AOT 1988 
test, or the AOT 1992 test. Along 
with test materials, the class lists 
were sent back to the schools with 
instructions assigning each student 
to a specific test. In addition, 
teachers were each sent a 
questionnaire concerned with 
assessing practice effects. 


The students who wrote the 1984 
achievement-over-time test are 
referred to hereafter as the AOT 84 


group, the students who wrote the 
1988 achievement-over-time test 
are referred to as the AOT 88 
group, and the students who wrote 
the 1992 test are called the AOT 92 
group to differentiate them from 
the students in the rest of the 
province who wrote the 1992 
regular achievement test. 


Before comparisons were made 
between the AOT 84 group and the 
students writing in 1984, and 
between the AOT 8&8 group and the 
students writing in 1988, it was 
necessary to verify that the 
students selected to participate 

in the special study were 
representative of students 
throughout the province. Since 
approximately one-third of the 
students sampled were assigned 


to the AOT 92 group, it was 


possible to compare their results 
with the provincial results. The 
large sample sizes allowed us to 


detect very small differences. Only 


differences where the probability of 


this difference occurring by chance 
was less than 0.01 were considered 


to be significant. Table 8-2 
presents the findings. At all three 
grades, the means of the AOT 92 
group and of the province were 


Table 8-2 
Comparison of AOT Sub-groups with Provincial Results 


AOT 92 Group ; 1992 Province | AOT 88 Group | 1992 Province | AOT 84 Group | 1992 Province 

1992 Items Actual 1992 Items Actual 1992 Items Actual 
Grade 3 Social Studies 
Number of 1992 Questions 50 50 24 24 25 25 
Raw Score Mean 38.7 38.0 18.5 18.5 19.5 19.3 
Standard Deviation 8.3 8.9 4.1 4.5 4.0 4.5 
Number of Students 856 31 599 856 31 599 857 31 599 
Grade 6 Language Arts 
Number of 1992 Questions 50 50 26 26 30 30 
Raw Score Mean 30.5 30.1 15.7 15.7 18.3 18.0 
Standard Deviation 8.5 8.5 47 4.7 5.8 5.7 
Number of Students 907 30 958 914 30 958 905 30 958 
Grade 9 Mathematics 
Number of 1992 Questions 49 49 31 31 30 30 
Raw Score mean 28.1 27.7 18.8" 17.4 16.6 16.4 
Standard Deviation 9.6 9.7 6.3 6.4 6.3 6.4 
Number of Students 11.3 24 522 954 24 522 1040 24 522 


within 0.4 raw score points. 

None of these differences was 
statistically significant at the 0.01 
level. As there were no significant 
differences between the means 

of the AOT 92 groups and the 
provincial means, we concluded 
that the samples of students in the 
special study were representative 
of all regular students in the 
province. 


However, as there were common 
questions across the three forms 
for a given subject, it was possible 
to carry out an additional check 
that the random assignment 
within the AOT group also resulted 
in sub-groups that were 
representative of the province. 


On the 1992 questions, the 
differences between the means of 
the students in the special study 
and the means of students in the 
rest of the province are within 0.3 
raw score points. These differences 
are not statistically significant at 
the 0.01 level. Based on these 
analyses, we concluded that the 
random assignment was successful 
for the Grade 3 Social Studies and 
the Grade 6 Language Arts and 
that these samples are 
representative of the province. 


The exception is the AOT 88 sub- 
group for Grade 9 Mathematics. 
This sub-group scored 1.4 points 
higher than its provincial 
counterpart. This difference is 
significant at the 0.01 level. The 
AOT 88 sub-group is significantly 


superior to both the AOT 92 sub- 
group and the 1992 Grade 9 
Mathematics population. The AOT 
84 sub-group is representative of 
the province. 


Where it was established that our 
samples were representative of the 
province, we were able to proceed 
with the comparisons between the 
1992 students in the study and the 
students in 1984 and 1988. As 
stated earlier, we restricted our 
sampling to schools with an 
estimated minimum size of 24 
students in the grade being 
sampled. When making 
comparisons with the previous 
years, we attempted to exclude 
students who wrote in the earlier 
years who were from schools that 
were smaller than the smallest 
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school size in the 1991 samples. 
However, individual student data 
were not available for 1984 Grade 3 
Social Studies and Grade 6 
Language Arts. It was not 
possible, therefore, to exclude from 
these comparisons students from 
schools smaller than the school 
sizes in the 1992 sample. 


Table 8-3 shows, for each grade, 
the number of common questions 
that were answered by both the 
1992 AOT students and the 
students in the previous years, the 
means and standard deviations for 
these questions, and the number of 
students on which these statistics 
were based. 


Differences between groups are not 
discussed at this stage as analysis 


of the practice effects can alter any 
conclusions drawn. More detailed 


analyses of the changes in 


achievement are described below 


for each grade. 


Practice Effects Study 


Design 


In 1992, a questionnaire was sent 
to teachers of the AOT groups 

to elicit information about student 
preparation practices. The study 
was designed to allow us to detect 
any inflation of the marks in the 
1988 AOT and 1984 AOT sub- 
groups due to student experience 
with 1988 or 1984 questions before 
they wrote the AOT 92 tests. 


Based on the teacher responses to 
the practice effects questionnaire, 


> 


students are divided into “practice’ 
and “no practice” groups. Students 
assigned to “practice” groups were 
reported to have used the 1988 

(or 1984 as appropriate) questions 
and answers to practice for writing 
the 1992 achievement exams. 


As with the main study, the means 
of 1992 questions allow us to verify 
the equal ability of groups of 
students in the “practice” and “no 
practice” groups. Differences in 
means for the 1988 or 1984 items 
allow us to detect a practice effect 
if it exists. The mean of the “no 
practice” group for each sub- 
sample is then compared with the 
respective 1988 or 1984 population 
and conclusions about achievement 
over time are drawn. 


Table 8-3 
Achievement-Over-Time Comparison of Descriptive Statistics 


AOT 88 Group Regular AOT 84 Group Regular 
Grade (wrote in 1992) 1988 Students | (wrote in 1992) | 1984 Students 

Grade 3 Social Studies 
Number of Questions 26 26 25 25 
Raw Score Mean 18.3 17.2 17.6 15.2 
Standard Deviation 5.2 5.0 4.7 5.0 
Number of Students 851 29 961 857 25 775 
Grade 6 Language Arts 
Number of Questions 24 24 20 : 20 
Raw Score Mean 15.4 15.2 13.7 13.6 
Standard Deviation 5.1 4.8 3.3 303 
Number of Students 914 32 069 915 38 812 
Grade 9 Mathematics 
Number of Questions 16 16 16 16 
Raw Score mean 12.6* 11.9 12.2 11.4 
Standard Deviation — 28 3.0 2.5 N/A** 
Number of Students 954 25 290 1040 27 121 


* The mean is taken from a subgroup that is significantly more able than that of the AOT 92. An equated mean of 11.1 would be 
considered more representative of the 1992 population. 


™* The original student response data for 1984 were not available. Question difficulties were used to reconstruct the means, but standard 


deviations could not be reconstructed. 
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Conclusions 


Social Studies 3 

Practice effects were found in both 
the 1988 AOT and 1984 AOT 
groups. Without this evidence, we 
could have concluded that 
achievement had improved since 
1988 and 1984. When practice 
effects are taken into account, 
however, achievement in 1992 has 
not changed significantly from 
1988 but has improved from that 
in 1984. This conclusion should be 
viewed with caution as the number 
of students reported to have 
practiced was only 204 from a 
sample of 523 students that could 
be identified from the practice 
effects questionnaire. 


Language Arts 6 

For the 1988 AOT group, a 
demonstrable practice effect was 
found among students who 
practiced on 1988 questions and 
answers compared to those who did 
not. There also appears to be a 
practice effect for the 1984 AOT 
sample, but the small number of 
students reported to have practiced 
casts doubt on this conclusion. In 
spite of evidence of practice effect, 
we can retain a conclusion of no 
change in achievement from 1988 
or 1984. 


Mathematics 9 

For the 1988 AOT group, the small 
number of students who practiced 
with answers, coupled with 
differences in ability level between 
the “practice” and “no practice” 
groups, leaves us unable to 
establish with certainty that a 
practice effect did occur. The lack 
of students who were reported to 
have practiced means there is no 
practice effect that would affect 
interpretation of the results. We 
conclude that achievement in 1992 
has not changed from 1988 but has 
improved over 1984. 


Written-Response Study 
Design 


Students’ writing performance has 
been of particular interest. to 
educators and to the public since 
the beginning of the Achievement 
Testing Program. Because of this 
interest, a study was initiated to 
compare 1992 writing with 1988 
writing and 1984 writing in Grade 6 
English Language Arts. This was a 
descriptive study that required 
teacher-readers to take a research 
or reader-as-observer look at the 
papers they read, rather than the 
usual evaluative or reader-as- 
assessor view of a teacher-marker. 
No attempt was made to rescore 
papers; rather, teacher-readers 
described features of the 1992, 
1988, and 1984 writing in the five 
scoring categories: 


Content 

—the effectiveness of ideas/ 
details/specifics chosen by the 
writer 

—how effectively the purpose is 
achieved 

—whether the reader’s interest 
has been captivated and 
maintained 


° Organization 

—focus 

~—coherent order 

—connections between events 
and/or details 

—closure 


*Sentence Structure 

—the degree to which the writer 
frames grammatically correct 
sentences 

—the effectiveness and variety of 
sentence type and length 


*Vocabulary 

—the effectiveness and accuracy 
of the words and expressions 
selected by the writer 


Conventions 
—mechanics 
-—grammar 


Comparisons were made at two 
standards: Acceptable (3), which 
represents work at an acceptable 
level for students completing 
Grade 6, and Excellent (5), which 
represents outstanding work for 
students completing Grade 6. 


Methodology 


Papers read in the study were 
selected at random from papers 
that received scores of Acceptable (3) 
or Excellent (5) on the June 1992, 
June 1988, and June 1984 
achievement tests. 


A group of experienced teacher- 
readers, representing all major 
regions of the province and a 
variety of school settings, reviewed 
the selected papers. 


Working alone, then in pairs, and 
then discussing the papers as a 
group, the teacher-readers 
described the papers for features of 
content, organization, sentence 
structure, vocabulary, and 
conventions. 


They then compared their 
descriptions of 1992, 1988, 1984 
Acceptable (3) and Excellent (5) 
papers to draw conclusions. 
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Results 


Acceptable (3) 

Teacher-readers felt that papers 
judged to be Acceptable (3) in 1992 
were not significantly different 


from 1984 Acceptable (3) papers. 
Although they felt that 1988 
Acceptable (3) papers were in the 
range of 1992 Acceptable (3) 
papers, they found both the 1992 


Acceptable (3) Papers in English Language Arts 6 


and the 1984 papers to be 
generally superior to the 1988 
papers. Teacher-reader comments 
on key features of these papers are 
summarized below. 


Key Features of 1992 
Acceptable (3) Papers 


Key Features of 1988 
Acceptable (3) Papers 


Key Features of 1984 
Acceptable (3) Papers 


Content 


Content 


Content 


*purpose established and sustained 


eclear idea of storyline development 


*relevant but predictable supporting details 


*purpose established but not generally 
sustained 


eiack of plot continuity; disjointed story lines 


evague details that do not support purpose 


*purpose established and sustained 


*sequential development; simple recounting 
of events 


*some supporting details evident 


Organization 


Organization 


Organization 


eintroductions generally either too sketchy 
or too complex 


eawareness of need to sequence events and 
details 


introductions often not attempted; writers 
assumed prompt was sufficient 


eawareness of need to sequence events; 
coherence weakened by inclusion of 
irrelevant details 


simplistic restatements of story prompt 
used as introductions 


eawareness of need to sequence events 
and details 


eweak or missing transitions were common in all three years 
*appropriate but abrupt closures were common in all three years 


Sentence Structure 


eattempt to imbed multiple ideas into 
sentences 


Sentence Structure 


*one idea per sentence 


In all three years, teachers noted: 
elack of variety of sentence type or length 


Sentence Structure 


*one idea per sentence 


*simple sentences predominate; few clauses or phrases used 
ewhen compound sentences occur, students tend to use the conjunction “and” 


Vocabulary Vocabulary 


| Vocabulary 


*specific words attempted *few vocabulary risks taken efew vocabulary risks taken 


estudents in all three years could use general words accurately 


Conventions Conventions 


| Conventions 


*spelling errors common; errors relatively 
phonetic 


*few spelling errors *few spelling errors 
ig 


*some problems with tense *some problems with tense egood handling of tense 


*pronoun-antecedent problems common *some pronoun-antecedent problems egood handling of pronouns and 


antecedents 


eproficient quotation mark usage in all three years 
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Excellent (5) 
Teacher-readers felt that papers 
judged to be Excellent (5) in 1992, 


1988, and 1984 were all within the 
same range. A summary of the 
teacher-reader comments on 


Excellent (5) Papers in English Language Arts 6 


significant differences and 
similarities in the five reporting 
categories follows. 


Key Features of 1992 
Excellent (5) Papers 


Key Features of 1988 
Excellent (5) Papers 


Key Features of 1984 
Excellent (5) Papers 


Content Content 


elack of plot plausibility common *appropriate ideas and details used to 


develop story lines 


einappropriate supporting details common *good use of supporting details 


Content 


eappropriate ideas and details used to 
develop story lines 


*good use of supporting details 


students in all three years were able to clearly establish a purpose for their writing 
«students in all three years captured reader interest by writing with imagination and creativity 


Organization Organization 


Organization 


In all three years, teachers noted: 
*introductions were present but generally weak 


sevents and details were arranged in a purposeful, effective order 


*transitions were generally used well 
eabrupt and/or contrived endings were common 


Sentence Structure | Sentence Structure 


Sentence Structure 


good variety of sentence type and length 
used regularly for effect 


*some variety of sentence type and length 
but frequent sentence fragment problems 
noted 


*some variety of sentence type and 
length 


eatfective, consistent control of sentences was seen in all three years 


Vocabulary Vocabulary 


Vocabulary 


*specific or descriptive verbs tended to 
‘be well used 


*specific or descriptive adjectives, verbs, 
and adverbs well used to describe settings 
and characters; elaborate, descriptive 
phrases also used 


*few vocabulary risks taken ewillingness to take vocabulary risks, but 


this led to some poor word choices 


specific or descriptive adjectives tended to 
be well used 


efew vocabulary risks taken 


Conventions Conventions 


Conventions 


*few spelling errors few spelling errors 


¢good handling of tense *good handling of tense 


*few spelling errors 


*inconsistent use of tense 


students in all three years consistently used quotation marks well 


compared to their 1984 and 1988 
counterparts. 


Concluding Comments 


The question that guided this study 
was: Has writing by Grade 6 
English Language Arts 
students improved from 1984 to 
1992? Teachers participating in 

the study felt that 1992 students at 
the Acceptable (3) and Excellent (5) 
levels of performance produced 
writing of similar quality when 


However, teacher-readers made 
several observations related to the 
changing nature of the writing over 
the three test years. 


The portrayal of adults in the 
students’ writing was markedly 
different on the three tests. In 


1984, adults were portrayed as 
either adversaries, such as 
poachers and criminals, or as 
responsible parents and law 
enforcers. These law-abiding 
adults were often utilized in the 
solution to whatever dilemma had 
emerged in the students’ writing, 
although many students used 
children to solve their story 
problem. Adults in 1988 papers 
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were mainly developed as 
adversaries or as dim-witted 
parents, and these adults were 
rarely used to help resolve story 
conflicts. Rather, 1988 writers 
provided solutions based on either 
the ineptitude of the criminals, the 
ingenuity of the children, or some 
form of magic. By 1992, adults 
were still seen as adversaries, but 
were rarely seen in other roles. 
These stories were most often 
resolved by a reliance on 
superhuman powers with which 
the children had been imbued. 
These children were easily able to 
out-run, out-think, out-fight, and 
“out-everything” all their 
adversaries. 


The type of prompt used on the 
three tests may have influenced 
the type of writing produced by 
students. In 1984 and 1992, the 
prompts placed two children in a 
realistic setting, whereas the 1988 
prompt provided greater potential 
for elements of fantasy to be 
incorporated into the response. 


The change in how students 
handled resolutions to their plot 
problems may be related to the 
story settings they tended to use. 
On all three tests, most students 
centred their stories on human 
adversaries; few monsters or 
supernatural creatures were noted 
in the writing. In 1984, these 
human adversaries usually 
wreaked havoc in a real-life 
setting. Many of the 1988 writers 
set their stories in a fantasy world 
in which obvious rules of cause and 
effect were nonetheless adhered to. 
A considerable number of 1992 
writers also used a fantasy setting, 
but one in which gaps in 
plausibility were more of the norm. 
Perhaps because they tended to 
have realistic settings more often, 
the 1984 papers also contained far 
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more unhappy or hopeless endings 
than did the 1988 or 1992 papers. 


One feature seen only in the 1992 
papers was that writers exhibited 
no sense of ownership, privacy, or 
personal property. Children in 
these stories went into strangers’ 
homes, rummaged through private 
articles in other people’s trunks, 
and used anything they wanted 
regardless of where they found it. 
In the 1984 and 1988 writing, 
children who helped to retrieve 
some valuables usually got to keep 
a reward. In the 1992 stories, 
children often got to keep 
everything they found and thereby 
became wealthy. 


While not in the realm of 
assessment, teachers also noted 
that penmanship quality has 
deteriorated over the three test 
years. As well, in writing from all 
three years, both themes and 
supporting details strongly 
refected the popular media 
influences of the time. Story lines 
and incidents from popular 
television shows, movies, and 
cartoons were often used as the 
basis for a plot. 


Finally, all teacher-readers felt 
that there had been a shocking 
change in the nature of the stories 
from 1984 to 1992. Teachers 
referred to the 1984 writing as 
“more child-like” and considerably 
‘less violent” than writing in 1992. 


Section 9 
Grade 6 
French Language Arts 


(Francais 6° année) 
Immersion Program 


v 


General Description 


The Grade 6 French Language Arts 
achievement test was administered 
to students in the French 
Immersion program. It was 
designed to reflect the writing and 
reading components of the program 
of studies: Programme d'études: Le 
francais & l’élémentaire: francais— 
immersion, Language Services, 
Alberta Education (1987). 


The test, Francais 6° année, had 
two parts, Part A: Writing and Part 
B: Reading, each worth 50% of the 
total test mark. 


Statistics are based on the results 
achieved by the 2055 Alberta 
students who wrote the test in June 
1992. This section of the report 
answers the following questions: 


* How many Grade 6 students 
wrote the test? ; 
¢ What percentage of Grade 6 
students who wrote the test 
attained the acceptable standard 
or higher in French Language 
Arts according to provincial 
criteria? 
What percentage of Grade 6 
students who wrote the test 
attained the standard of 
excellence or higher according to 
provincial criteria? 
What did Grade 6 students know 
and what could they do in 
French Language Arts? 
What parts of the French 
Language Arts curriculum 
caused Grade 6 students 
difficulty? 


Summary of Results 


Results show that 83.2% of 
students who wrote the test 
achieved the acceptable standard 
or higher and 12.6% achieved the 
standard of excellence or higher on 
the total test. 


Content of the Test 


Part A: Writing consisted of one 
writing assignment, which 
provided the beginning of a story 
and an illustration. Students had 
to continue and complete the story. 


The assignment set a specific 
writing task but allowed students 
a lot of freedom in choosing what to 
write and how to write it. Students 
had 80 minutes in the morning to 
do the writing assignment. Markers 
scored the paper on five dimensions: 
Content, Organization, Sentence 
Structure, Vocabulary, and 
Conventions. 


Part B: Reading consisted of 50 
multiple-choice questions based on 
eight reading selections. Students 
had one hour in the afternoon to do 
this part. Reading selections were 
chosen to reflect the literary genres 
prescribed in the Program of 
Studies and to reflect the interests 
and reading level of Grade 6 
students. They included Canadian 
material, and all selections met 
Alberta Education’s criteria for 
Tolerance and Understanding. 
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Test Blueprint 


Table 9-1 presents the blueprint 
used to develop Part A: Writing of 
the Grade 6 French Language Arts 
achievement test. It describes the 
writing assignment, the scoring 
categories, and the allocation of 
marks. 


Table 9-1 

Grade 6 French Language Arts 
Immersion Program 
Achievement Test Blueprint 
Part A: Writing 


dune 1992 
Reporting Category (Scoring Guide)* Description of Writing Assignment Range of Marks 
Content** (Selecting details to achieve a purpose) 
Events and/or actions should be plausible and appropriate to 
the student's purpose for communicating. The student should 
be able to describe characters and settings that are 
appropriate within the context of terms of references The writing assignment follows a writing 5—Excellent 
established by the student. prompt. The assignment allows the student | 4—Proficient 
to select the format that will best fit his/her 3—Satisfactory 
Organization** (Organizing details into a coherent whole) approach to the prompt. The student has to | 2—Limited 
The student should be able to place events in a coherent continue and complete a story. The writing 1—Poor 


order. 


Sentence Structure (Structuring sentences effectively) 
The student should be able to use a variety of sentence 
structures effectively in writing. 


Vocabulary (Selecting words and expressions correctly and 
effectively) 

The student should be able to use words and expressions 
effectively in writing. 


Conventions (Using the conventions of language correctly 
and effectively) 

The student should be able to communicate clearly in writing 
by adhering to appropriate spelling, grammar, punctuation, and 
capitalization. 


task is specific but allows the student much 
latitude in the choice of details, actions, 
and events 


INS—Insufficient 


“These reporting categories are based on the Program of Studies for French Language Arts (French Immersion), Language Services, 


Alberta Education (1987) 


“* These two categories are each weighted to be worth twice as much as each of the other three. 
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Table 9-2 presents the blueprint 
used to develop Part B: Reading of 
the Grade 6 French Language Arts 
achievement test. It shows the 
distribution of questions according 
to the curricular elements (skills) 
and according to the cognitive level 
being assessed. 


Table 9-2 

Grade 6 French Language Arts 
Immersion Program 
Achievement Test Blueprint 
Part B: Reading 

dune 1992 


Reporting Category* 


Question Numbers by Cognitive Level 


identifying and 
Selecting 


Regrouping 
and Inferring 


Evaluating 
and Judging 


Number of 
questions 


% of 
test 


the reader 


1. Relationship between the author and 


The student should be able to identify, to 
infer, and to evaluate the author's purpose 
and the means used in the text (form and 
style) to achieve it. 


1, 6, 20 


19, 24, 26, 
39, 50 


16 


2. Associating meanings 
The student should, based on his/her 
knowledge and on the given context, be 
able to understand a reading selection by 
identifying, inferring, or evaluating 
details given about characters or events 
(actions, motives, values, conflicts, etc.). 
Also, the student should be abie to 
understand words and expressions in 
context. 


11, 16, 28, 
29, 33, 34, 
35, 37, 40 


2,3, 4,9, 
15, 17, 18, 
21, 22, 23, 
36, 43, 46, 
49 


10, 13, 27, 
41, 42, 44, 
45, 47 


31 


62 


3. Main ideas, opinions, and conclusions 


The student should be able to identify, infer, 


and evaluate the information within the 
reading selection to understand its 
meaning, to understand expressed 
opinions, to deduce the main idea, and to 
predict possible outcomes or conclusions. 


Number of questions 


30, 38 


11 


% of test 


22 


5, 14, 32, 
48 


21 
42 


7, 8, 12, 
25, 31 


18 
36 


11 


50 


22 


100 


* These reporting categories are based on the Program of Studies for French Language Arts (French Immersion), Language Services, 


Alberta Education (1987) 
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Student Participation 


Students who were enrolled in 
Grade 6 French Language Arts 
could write the achievement test. 
Participation was decided by each 
jurisdiction; 2055 students wrote 
the 1992 test. Table 9-3 presents 
the number and percentage 
distribution of students who wrote 
the Grade 6 French Language Arts 
achievement test. 


Table 9-3 


Grade 6 French Language Arts 
Immersion Program 

Student Participation 

dune 1992 


Category 


Number 
of Students 


Percentage 
of Students 


Students Absent 


Students Present but not Participating: 


66 


3.1 


0.3 
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Results in Relation to 
Standards 


Figures 9-1 and 9-2 and Table 9-4 
show the number and the 
percentage of students achieving 
the acceptable standard or higher 
and the standard of excellence or 
higher on the total test and on 
components of the test. 


Figure 9-1 

Grade 6 French Language Arts 

Immersion Program 

Percentage of Students Achieving Acceptable Standard or 
Higher on the Total Test and on Components of the Test 
June 1992 


100 


80 


60 
% 


Total Test Reading Skills 


Y Achievement Standard* 
Vs 4 (See Appendix A) 
* the percentage of students in the province expected to meet the acceptable standard or 


higher 
** the percentage of students in the province who met the acceptable standard or higher 


Actual Results** 


SE es 


Figure 9-2 

Grade 6 French Language Arts 

Immersion Program 

Percentage of Students Achieving Standard of Excellence or 
Higher on the Total Test and on Components of the Test 
June 1992 


25 
20 


Total Test i Reading Skills 


WY Achievement Standard” Actual Results** 


ZA (See Appendix A) 


* the percentage of students in the province expected to meet the standard of excellence or 
higher 
** the percentage of students in the province who met the standard of excellence or higher 
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Table 9-4 
Grade 6 French Language Arts 
Immersion Program 


Students Achieving Standards on the Total Test and on Components of the Test 


June 1992 
Provincial Students Achieving Assessment 
Maximum | Assessmen Standard or Higher 
Possible Standard* Expected Actual 

Reporting Category Score (Raw Score’ Number Number 
Standard of Excellence or Higher 
Total Test*** 100 80 308 258 
Writing Only 35 28 308 415 
Reading Skills Only 50 40 308 334 
Acceptable Standard or Higher 
Total Test*** 100 51 1 747 1709 
Writing Only 35 18 1747 1709 
Reading Skills Only 50 25 1747 1 544 
Below Acceptable Standard N/A N/A N/A 134 
on Both Components 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the lowest score a 
student must achieve for his/her performance to be judged “acceptable” and/or “excellent” in relation to curricular expectations. 


See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provincial Assessment 


Standard. See Appendix A. 


*““The Writing score is multiplied by 50/35 before being added to the Reading score so that both components are weighted equally. 


Levels of performance on the test 
were usually close to expectations. 
Slightly fewer students than 
expected met the standard of 
excellence on the total test, and 
slightly more met the standard of 
excellence for each component. The 
percentage of students meeting the 
acceptable standard for the total 
test and for written response were 
very close to expectations, but 
nearly 10% fewer students than 
expected met the acceptable 
standard for reading skills. 
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The number of students achieving 
or exceeding the acceptable 
standard and the standard of 
excellence for each jurisdiction was 
analyzed to determine whether 
jurisdictions were below 
expectations, meeting expectations, 
or above expectations. Jurisdictions 
classified as meeting expectations 
were those for whom the difference 
between the actual number of 
students and the expected number 
of students at or above expectations 
was not statistically significant. A 
95% confidence interval was used; 


a aca aaa a eS 


this criterion means that differences 
are only reported when there is a 
5% or smaller probability that a 
difference of that size would occur 
by chance. 


The results are reported in Table 
9-5. The percentages are based on 
39 jurisdictions (including private 
schools). One jurisdictions was too 
small to give reliable results. 


Table 9-5 

Grade 6 French Language Arts 

Immersion Program 

Percentage Distribution of Jurisdictions Meeting Achievement Standards on the Total Test 
and on Components of the Test 


June 1992 
Percentage Distribution of Jurisdictions 
Not Meeting Provincial | Meeting Provincial | Exceeding Provincial 
Achievement Achievement Achievement 
Reporting Category Standard (Per Cent) | Standard (Per Cent) | Standard (Per Cent) 

Standard of Excellence or Higher 

Total Test 5.1 92.3 2.6 

Writing Only 0.0 76.9 23.1 

Reading Skills Only 51 79.5 15.4 
Acceptable Standard or Higher 

Total Test 23.1 71.8 5.1 

Writing Only 5.1 89.7 5.1 

Reading Skills Only 46.2 53.8 0.0 


Results for context of the scoring descriptors. second set of scores was available 
Part A: Writing Complete scoring guides are to confirm scoring consistency. Of 
available from the Student the scores awarded on the second 
Haw scores were calculated by Evaluation Branch, Alberta reading, 88.5% were identical to 
adding the marks earned for each Education. All participating the original score on the same scale 
of the five reporting CAleennes schools should have extra copies of or varied by only one point. It is 
The Content and Organization Part A: Writing booklets to use in important to note that the one- 
categories had a weighting of 10 conjunction with information marker system produces results 


marks each and the Sentence 
Structure, Vocabulary, and 
Conventions categories had a 


provided in this Provincial Report. that are reliable for groups of 25 or 
more students. Achievement test 


‘cht 5 k h Although the papers were scored scores are not intended to be 
WEts PEE OPS nese eee: on a one-marker system, 96 papers reliable for individual students. 
a randomly selected among all 
ce as ee ee a papers written by French Statistics in Table 9-6 are based on 
Mids Sean DOr reopen He Immersion and Francophone results achieved by 2055 students. 


context of the assignment to which 


students were re-marked so that a 
students responded and in the 


Table 9-6 


Grade 6 French Language Arts 

Immersion Program 

Percentage Distribution of Scores by Reporting Category 
Part A: Writing 

June 1992 


Reporting Category 


Sentence 
Scoring Scale Content Organization Structure Vocabulary Conventions 


(2) Limited 12.7 14.6 10.9 12.0 23.5 
(1) Poor 1.8 2.1 1.2 0.9 3.7 
(INS) Insufficient 0.1 0.1 0.1 0.1 0.1 
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Results for multiple-choice question. The the jurisdiction and school reports. 


Part B: Reading correct response for each question Variations in patterns of students’ 
Table 9-7 shows the percentage is also given. The results shown in responses to questions can help to 

Table 9-7 can best be used in indicate strengths and weaknesses 
of seadentew 1 chore ach conjunction with similar tables in in local educational programs. 


alternative (A,B,C, and D) for each 


Table 9-7 
Grade 6 French Language Arts 
Immersion Program 
Results for Individual Multiple-Choice Questions* 
dune 1992 
Distribution of Responses (%) Distribution of Responses (%) 
Item Key A B c D Item Key A B Cc D 
1 A 77.7 4.4 3.1 14.6 26 A 56.8 8.7 27.4 6.3 
2 D 5.6 17.6 10.8 65.9 27 D 12.5 6.9 6.9 73.5 
3 D 2.6 5.7 10.1 81.5 28 A 54.2 11.6 12.6 21.3 
4 A 71.3 11.4 12.7 4.5 29 Cc 8.0 20.7 57.3 13.6 
5 B 20.6 60.2 12.4 6.8 30 D 10.0 6.2 411.6 71.8 
6 Cc 8.5 5.2 64.5 21.4 31 A 63.9 7.6 18.1 10.0 
7 D 20.0 15.0 5.0 59.6 32 Cc 12.8 10.3 68.8 7.6 
8 B 5.1 83.7 6.9 3.9 33 B 22.6 51.2 16.9 8.4 
9 Cc 35.8 17.7 38.3 7.5 34 A 88.7 6.2 2.3 2.2 
10 D 9.3 15.9 * 28.5 45.9 35 Cc 20.8 48 64.5 9.4 
11 B 18.0 44.4 22.6 14.7 36 B 16.4 54.1 24.2 4.9 
12 A 57.0 13.6 16.6 12.5 37 Cc 3.9 7.2 82.9 5.6 
13 B 16.7 70.8 6.6 5.6 38 Cc 8.4 10.5 73.2 75 
14 Cc 10.0 11.0 64.9 13.9 39 B 6.7 57.6 8.3 26.2 
15 A 37.2 31.7 25.5 5.1 40 Cc 7.5 74 75.1 91 
16 A 54.2 36.1 3.9 5.7 41 D 11.0 10.1 4.3 73.7 
17 D 7.5 2.2 5.0 85.1 42 Cc 75 19.4 67.0 49 
18 D 6.7 4.4 9.2 79.6 43 A 83.6 4.3 6.0 5.0 
19 Cc 5.7 6.6 81.2 6.3 44 D 16.0 12.3 16.6 53.6 
20 D 11.4 4.3 19.0 65.0 45 B 13.5 36.5 40.8 7.4 
21 B 18.0 47.5 12.8 21.5 46 B 9.8 30.3 9.0 49.2 
22 B 20.5 59.1 10.5 9.5 47 A 49.9 27.4 11.0 9.6 
23 Cc 6.2 4.6 76.2 12.7 48 A 54.0 10.1 18.6 15.0 
24 D 18.5 15.9 11.1 54.2 49 A 29.7 9.7 49.0 9.4 
25 D 8.6 7.2 15.6 68.3 50 Cc 22.5 14.9 39.5 20.9 


*The sum of the percentages for each question may be less than 100% because the No Response category is not included. 


Examiner’s Observations Their writing performance limited. Certain common errors of 
Part A: Writing supports teacher comments that immersion students—e.g., “Je faut 
Wachee aadiatnd with there is more writing going on faire ...”, or disregard for 
marking were generally satisfied in the classroom. Content, gender,—appear to have become a 
with the overall quality of Organization, Sentence Structure, part of many students’ language 
students’ writing. Students ay emma in ane were patterns. 
: ‘ : judged to be of good quality. | 
oyousty Geno ne oe Students continue to have Markers had the following 


experience of reading and writing 
stories in class in order to complete 
the assignment. They presented 


difficulty with Conventions. In this comments: 
category, fewer students achieved 


sufficient content well enough:to the level of excellence and a ¢ Students know that their writing 
maintain dhoweadeteibarcat greater number were scored as will be read and they write with 
a purpose. 
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* The Program of Studies 
emphasizes reading and writing, 
and students are writing more 
and with better quality. 
Students continue to have a 
weak grasp of conventions. Some 
aspects of conventions that 
should have been mastered by 
the sixth grade are not. 


Part B: Reading 

Markers and standard setters 
thought that the reading portion of 
the test represented an appropriate 


reading level for Grade 6 students. 
The variety and quality of reading 
selections, the Canadian content, 
and the themes dealt with in the 
readings were also judged to be 
very appropriate for Grade 6 
students. Although students did 
very well on the test as a whole, 
not enough students met the 
acceptable standard in reading. 
Reading is such an essential skill 
that students need to continue to 
develop an appreciation for reading 
and reading comprehension 


throughout their schooling. 


Sample questions are given below 
to highlight the strengths and 
weaknesses of students achieving 
at the acceptable standard and at 
the standard of excellence. For 
each sample question, the correct 
answer is marked with an asterisk 
and the percentage of students 
choosing each alternative is given 
(these figures do not add up to 100 
because a few students did not 
answer the question). 


13. Lacoccinelle utilise ses deux ailes 
cuirassées surtout pour 


16,7 A. se glisser 

70,8 *B. se protéger 

6,6 C.  éviter la sécheresse 

5,6 D. vivre sous des tas de feuilles 


48. D'aprés l'auteur, la recette miracle qui 
a changé Timothée était 


54,0 *A. son accueil dans une famille de dix 
enfants 

10,1 B. sa fierté et son indépendance 

18,6 C. son cours de dressage 

15,0 D. son intelligence 


Acceptable Standard—Sample Questions and 


Commentary 


Question 13 required students to judge, among four possible 
alternatives, the principal importance of the ladybird beetle’s 
armoured wings according to the information given in the passage. 
Most students achieving at the acceptable standard were able to 


choose the correct answer. 


Question 48 asked students to infer one of the main ideas of the 
passage. Many students performing at the acceptable standard had 


difficulty with this question. 


The strengths of students who achieved at the acceptable standard 


include 


*the ability to identify and select main ideas and supporting 


details 


*the ability to make inferences and judgments of a relatively 


simple nature 


Many students achieving at the acceptable standard had difficulty 
*identifying and selecting main ideas and supporting ideas when 


these were not obvious 


making inferences and judgments that required deeper 


thinking 
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22. Quand le poéte dit: “Chacun tracera sa 
voie” (vers 10), il veut dire que chaque 
personne 


20,5 A. veut connaitre son avenir 

59,1 *B. suit son propre chemin 

10,5 C. décide ot elle va vivre 
9,5 D. part ala découverte 


45. Timothée était un candidat difficile a 
Padoption surtout parce qui’il était 


13,5 A. trop fier 

36,5*B. mal éléve 
40,8 C. malpropre 
7,4 D. trop jeune 
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Standard of Excellence—Sample Questions and 
Commentary 

Question 22 required students to identify the correct meaning of an 
expression in a poem. Most students achieving the standard of 
excellence were able to select the right answer. 


Question 45 asked students to infer and to evaluate an important idea 
in the reading selection. Some students achieving the standard of 
excellence had difficulty selecting the correct answer. Too many 
students chose alternative C, which gave a minor, not the main reason. 


Concluding Comments 


The test results in this report show that student achievement in 
French Language Arts is generally satisfactory. However, too many 
students did not meet the acceptable standard for reading. Continued 
attention to reading comprehension would help to improve these 
results. 


Section 10* 


Francais 6 année 
Programme francophone 


Description générale 

Le Test de rendement, Francais 6° 
année, a été administré aux éléves 
des classes francophones. Ce test 
refléte les composantes «Ecriture» 
et «Lecture», Le francais a 
Vélémentaire: frangais—langue 
maternelle: programme d'études, 
Language Services, Alberta 
Education (1987). 


Le test comportait deux parties, 
Partie A: Production écrite et Partie 
B: Lecture. Chaque partie valait 
50% de la note totale du test. 


L'information présentée dans cette 
section est basée sur les résultats 
des 200 éléves des classes 
francophones qui ont écrit ce test 
en juin 1992. Cette section du 
rapport donne des réponses aux 
questions suivantes: 


*¢ Combien d’éléves de 6° année ont 
écrit le test? 

® Quel pourcentage des éléves de 
6° année qui ont écrit le test ont 
atteint ou surpassé le standard 
acceptable selon les critéres 
provinciaux? 

* Quel pourcentage des éléves de 
6° année qui ont écrit le test on 
atteint ou surpassé le standard 
d’excellence selon les critéres 
provinciaux? 

* Quelles connaissances et quelles 

habiletés en francais avaient les 

éléves de 6° année? 

Quelles parties du programme de 

francais ont causé des difficultés 

pour les éléves de 6° année? 


Résumé des résultats 


Les résultats montrent que, pour le 
test en entier, 87,5% des éléves qui 
ont écrit le test ont atteint ou 
surpassé le standard acceptable et 
24,5% ont atteint ou surpassé le 
standard d’excellence dans le test 
en entier. 


Contenu du test 


Dans la Partie A: Production écrite, 
on demandait aux éléves de 
compléter une histoire dont on 
donnait le commencement. Le 
début de Vhistoire donné était 
accompagné d’une illustration. 


L’examen présentait une tache 
précise de production écrite mais 
laissait 4 l’éléve une grande 
latitude dans la sélection du 
contenu et de la facon de le 
présenter. Les éléves avaient 80 
minutes le matin pour faire leur 
production écrite. Les correcteurs 
ont noté les productions écrites sur 
cing domaines: contenu, 
développement, structures de 
phrases, vocabulaire et usage. 


La Partie B: Lecture comportait 50 
questions a choix multiple basées 
sur huit textes de genres littéraires 
différents. Les éléves avaient une 
heure l’aprés-midi pour compléter 
cette partie. Les textes de lecture 
ont été choisis pour représenter les 
différents genres littéraires au 
programme et aussi selon les 
intéréts et niveaux de lecture des 
éléves de 6° année. 


Plusieurs sélections étaient des 
textes canadiens et tous ont 
rencontré les exigences de 
tolérance et compréhension de 
Alberta Education. 


* An English translation of this section of the report is available upon request by calling Mr. Dennis Belyk, Assistant Director, 


at 427-0010. 


Schéma directeurs du 
Test de rendement 


Le tableau 10-1 présente le schéma 
utilisé pour développer la Partie A: 
Production écrite du Test de 
rendement, Francais 6° année. Ce 
schéma décrit les domaines 
d’évaluation, la tache et le baréme 
de notation. 
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Tableau 10-1 
Francais 6° année 
Programme francophone 
Schéma du Test de rendement 
Partie A: Production écrite 
Juin 1992 


Domaine d’évaluation* 


Description de tache 


Bareme de notation 


Contenu* (choisir des détails pour atteindre un but) 

Les événements et/ou actions devraient étre plausibles et 
appropriés a |'intention de communication de |’éléve. L’éléve 
devrait pouvoir choisir des détails pour décrire des 
personnages et un cadre qui soient appropriés dans le 
contexte qu’il a étabii. 


Développement** (organiser les détails en un tout cohérent) 
L’éléve devrait pouvoir placer les événements dans un ordre 
coherent. 


Structures de phrases (bien structurer les phrases) 
L’éléve devrait pouvoir bien employer diverses structures de 
phrases par écrit. 


Vocabulaire (bien choisir et bien employer les mots et 
expressions) 

L’éléve devrait pouvoir employer les mots et expressions 
appropriés par écrit. 


Usage (bien employer les conventions de la langue) 
L’éléve devrait pouvoir communiquer clairement par écrit en 


ponctuation et des majuscules. 


respectant les régles de l’orthographe, de la grammaire, de la 


L’exercice consiste a continuer une histoire 
dont le commencement est donné. I! s’agit 
d’une tache précise, mais qui permet a 
'éléve une grande latitude dans te choix 
de détails, d’actions ou d’événemenis. 


5—Excellent 
4—Compétent 
3—Satisfaisant 
2—Limité 
1—Pauvre 
INS—Insuffisant 


*Ces cing catégories pour le rapport se trouvent dans la section «Ecriture», Le francais a I’élémentaire: francais—langue matemelle: 
programme d'études, Language Services, Alberta Education (1987). 


** Chacune de ces deux catégories a une valeur deux fois plus grande que chacune des trois autres catégories. 
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Le tableau 10-2 présente le schéma 
utilisé pour développer la Partie B: 
Lecture du Test de rendement, 
Francais 6° année. Ce schéma 
montre la répartition des questions 
selon les éléments/habiletés au 
programme et selon les niveaux 
cognitifs évalués. 


Tableau 10-2 

Francais 6° année 
Programme francophone 
Schéma du Test de rendement 
Partie B: Lecture 

Juin 1992 


Niveau cognitif % 


Identifier et | Regrouper et Evaluer et Nb. de du 
Eléments/Habiletés* sélectionner inférer juger questions test 


1. Rapport entre l’auteur et le lecteur 
L’éléve devrait étre capable d’identifier, 
d'inférer et d’évaluer I’intention de 
communication de l’auteur et d’établir un - 

rapport entre cette intention et les moyens 

utilisés dans le texte (forme et style) pour 
la transmettre. 


1,6, 20 19, 24, 26, 8 16 
39, 50 


. Association des idées et des détails 
L’éléve devrait étre capable, a partirde ses; 11, 16, 28, 2, 3, 4, 9, 10, 13, 27, 31 62 
connaissances et en tenant compte du 29, 33, 34, 15, 17, 18, 41, 42, 44, 
contexte, de saisir le sens d’un passage 35, 37, 40 21, 22, 23, 45, 47 

en identifiant, en inférant ou en évaluant 36, 43, 46, 
les détails portant sur les personnages ou 49 

sur les faits (actions, motifs, valeurs, 
conflits, etc.). De plus, I’éléve devrait étre 
capable de déceler le sens des mots 

et des expressions, selon le contexte. 


3. Idées principales, opinions et 
conclusions 
L'éléve devrait étre capable d’identifier, 30, 38 5, 14, 32, 7, 8, 12, 11 22 
d'inférer et d’évaluer les idées du texte 48 25, 31 
présenté (le fond) afin de saisir le sens, 
d’en déduire l’idée principale, de saisir les 
opinions exprimées, ou de prédire les 
suites possibles ou la conclusion. 


Nombre de questions 11 21 18 50 
% du test 22 42 36 100 


*Ces Eléments/Habiletés se trouvent dans la section «Lecture», Le francais a I’élémentaire: francais—langue maternelle: programme 
d'études, Language Services, Alberta Education (1987). 
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Participation 

Les éléves qui suivaient les cours 
de Francais 6° année pour 
francophones pouvaient écrire ce 
test. Chaque juridiction scolaire 
devait décider de sa participation; 
200 éléves ont écrit le test de 1992. 
Le tableau 10-3 présente le nombre 
et le pourcentage d’éléves qui ont 
écrit le Test de rendement de 
Francais 6° année. 


Tableau 10-3 

Francais 6° année 
Programme francophone 
Participation des éléves 
Juin 1992 


Nombre 
Catégorie d’éleves 


Pourcentage 
d’éleves 


Elaves absents 


Eléves présents, mais qui n'ont pas participé 
Li ae ee 2 


Esso si ation i 


oe 


100 


Résultats par rapport 
aux standards 


Les figures 10-1 et 10-2 et le 
tableau 10-4 montrent le 
pourcentage d’éléves qui ont 
atteint ou surpassé le standard 
acceptable et le standard 
d’excellence dans le test en entier 
et dans les parties du test. 


Figure 10-1 

Francais 6e année 

Programme francophone 

Pourcentage des éléves qui ont atteint ou surpassé le standard 
acceptable dans le test en entier et dans les parties du test 
Juin 1992 


100 
80 
60 
% 
40 
20 
0 
Total Production écrite Lecture 
F Standard de rendement* Résultats actuels** 
|] (Voir Annexe A) 


*le pourcentage des éléves dans la province qu'on s’attend pouvoir atteindre ou surpasser 
le standard acceptable 
**le pourcentage des éléves dans la province qui ont atteint le standard 


Figure 10-2 

Francais 6 année 

Programme francophone 

Pourcentage des éléves qui ont atteint ou surpassé le standard 
d’excellence dans le test en entier et dans les parties du test 
Juin 1992 


40 


Lecture 


Résultats actuels** 


~//) Standard de rendement* 
YY (Voir Annexe A) 


*le pourcentage des éléves dans la province qu’on s’attend pouvoir atteindre ou surpasser 
le standard d’excellence 
**le pourcentage des éléves dans la province qui ont atteint le standard 
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Tableau 10-4 
Francais 6 année 
Programme francophone 


Etudiants qui ont atteint les standards 


Juin 1992 
Standard Etudiants qui ont atteint ou 
provincial | surpassé le standard d’évaluation 
Maximum d’évaluation* Nombre Nombre 
Catégorie pour le rapport possible (note brute) attendu actuel 

Standard d’excellence on plus 

Total*** 100 80 30 49 

Production écrite seulement 35 28 30 70 

Lecture seulement 50 40 30 54 

Standard acceptable ou plus 

Total*** 100 51 170 175 

Production écrite seulement 35 18 170 185 

Lecture seulement 50 25 170 157 

Au dessous du standard acceptable Ne s’applique | Ne s’applique Ne s’applique 7 

sur les deux composantes pas pas pas 


* Le standard provincial d’évaluation est une note établie par des procédures appropriées d’établissement de standards et représente la plus 
basse note que |’éléves peut atteindre pour que son rendement soit jugé «acceptable» ou «excellent» en fonction des attentes du 
programme. Voir Annexe A. 


** Le standard provincial de rendement est le pourcentage d’éléves qu’on s’attend pouvoir atteindre ou surpasser le standard provincial 
d’évaluation. Voir Annexe A. 


***La note de la Production écrite est muitipliée par 50/35 avant qu'elle soit additionnée a la note de la Lecture de sorte que les deux | 
composantes représentent une préportion égale. 


A cause du petit nombre de 
juridictions impliquées dans 
Yévaluation francophone, nous ne 
rapportons aucune analyse des 
résultats par juridiction comparée 
aux attentes. 


Les niveaux de rendement sur le 
test ont généralement dépassé les 
attentes. Un peu moins d’éléves 
que prévu ont atteint le standard 
acceptable en lecture. 
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Résultats pour la 
Partie A: Production 
écrite 

On a calculé les notes brutes en 
additionnant les notes attribuées a 
chacun des cing domaines 
d’évaluation. Les catégories 
Contenu et Développement 
valaient 10 points chacune et les 
catégories Structures de phrases, 
Vocabulaire et Usage valaient 

5 points chacune. 


Les résultats pour la Partie A: 
Production écrite sont le mieux 
compris dans le contexte de la 
tache présentée 4 l’éléve et dans le 


contexte des descripteurs qui font 
partie du baréme de notation. Les 
guides de notation sont disponibles 
du Student Evaluation Branch, 
Alberta Education. Les écoles 
devraient avoir des copies 
supplémentaires du livret 
d’examen de la Partie A: 
Production écrite pour utiliser avec 
Yinformation présentée dans ce 
rapport provincial. 


La production écrite a été notée 
par un seul correcteur. Cependant 
96 copies choisies au hasard parmi 
toutes les copies écrites par les 
éléves des programmes 
francophones et d’immersion 


francaise ont été notées par un 
deuxiéme correcteur pour fournir 
un moyen de confirmer la 
compatibilité des notes; 88,5% des 
notes attribuées lors de la 
deuxiéme notation étaient soit 
identiques 4 la note originale dans 
la méme catégorie ou bien 
variaient par seulement un point. 
Il est important de remarquer que 
la notation par un seul correcteur 
donne des résultats fidéles pour 
des groupes de 25 étudiants ou 
plus. Les Tests de rendement ne 
sont pas concus pour donner des 
résultats fidéles pour des 
individus. 


Tableau 10-5 

Francais 6° année 
Programme francophone 
Distribution en pourcentage des notes par domaine d’évaluation 


Juin 1992 


Baréme de notation 


Contenu 


Domaine d’évaluation 


Développement 


Structures de 
phrases 


Vocabulaire 


Usage 


ou Copie blanche 


(2) Limité 9,5 11,0 6,0 7,5 13,0 
(1) Pauvre 0,5 0,5 0,5 1,5 
(INS) Insuffisant 0 0 0 0 


Résultats pour la Les résultats présentés dans 
Partie B: Lecture le Tableau 10-6 sont mieux 
interprétés si on examine en méme 
temps les rapports de l’école ou de 
la juridiction et les textes et les 
questions au Test de rendement 
lui-méme. Ceci donnerait un bon 
apercu des forces et des faiblesses 
au niveau local. 


Dans la Partie B: Lecture, chacune 
des 50 questions 4 choix multiple 
avait une valeur de un point. Le 
tableau 10-6 indique, pour chaque 
question, la bonne réponse ainsi 
que le pourcentage d’éléves qui ont 
choisi chaque réponse possible 

(A, B, C, D). 


Tableau 10-6 

Francais 6° année 

Programme francophone 

Résultats pour chaque question a choix multiple* 


duin 1992 
Distribution des réponses (%) Distribution des réponses (%) 
Question | Clé A B c D  {Question] clé A B c pi 
1 A 64,0 4,5 2,5 29,0 26 A 56,0 8,0 28,0 7,0 
2 D 10,0 27,0 9,0 54,0 27 D 8,0 8,0 7,0 77,0 
3 D 2,0 1,0 1,5 95,5 28 A 57,0 18,0 12,5 12,0 
4 A 83,0 5,5 10,0 1,0 29 Cc 9,0 17,0 64,0 9,5 
5 B 16,0 63,0 15,0 6,0 30 D 8,5 6,5 6,0 79,0 
6 Cc 5,5 4,0 76,0 14,5 31 A 72,5 6,5 12,5 8,5 
7 D 31,5 14,0 3,0 51,5 32 Cc 9,0 6,5 81,5 3,0 
8 B 4,5 77,5 12,0 5,0 33 B 22,5 54,5 11,0 10,0 
9 Cc 20,5 13,5 64,0 2,0 34 A 87,0 4,5 4,5 2,5 
10 D 6,5 22,0 23,0 48,0 35 Cc 19,5 3,0 67,5 9,0 
11 B 17,0 48,0 23,0 12,0 36 B 21,5 56,0 14,5 7,0 
12 A 42,5 23,0 21,0 13,5 37 Cc 4,0 4,5 87,5 3,0 
13 B 15,0 69,0 10,0 5,0 38 Cc 6,5 8,0 75,5 9,0 
14 Cc 6,5 7,5 76,0 10,0 39 B 4,0 77,0 5,0 12,5 
15 A 43,0 33,5 17,5 5,5 40 Cc 8,0 11,0 72,0 8,5 
16 A 59,5 29,0 4,5 7,0 41 D 12,5 11,0 2,0 74,0 
17 D 7,5 5,5 6,5 80,5 42 c 11,5 18,0 67,5 2,0 
18 D 7,0 4,0 9,0 80,0 43 A 84,5 2,5 6,5 5,5 
19 Cc 4,0 4,5 82,5 8,0 44 D 29,5 7,0 10,0 52,0 
20 D 12,5 2,5 13,5 71,0 45 B 8,5 57,5 28,0 5,5 
21 B 15,5 43,5 11,0 29,5 46 B 11,0 32,5 5,0 51,0 
22 B 10,0 68,5 9,0 12,5 47 A 47,5 27,0 12,5 12,5 
23 c 5,0 3,5 81,5 9,5 48 A 62,5 13,0 16,0 8,0 
24 D 11,5 12,0 5,5 70,5 49 A 27,0 10,0 52,5 10,0 
25 D 10,5 7,0 14,0 67,5 50 Cc 17,0 18,0 44,5 20,0 


*La somme des pourcentages pour chaque question peut étre moins de 100 parce que la catégorie «pas répondu» n'est pas incluse. 
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Commentaires de 
Vexaminateur 


Partie A: Production écrite 

Les enseignants qui ont participé 
aux corrections ont été bien 
impressionnés par la qualité des 
productions écrites des éléves. I 
est évident que les éléves ont su 
tirer avantage de leurs expériences 
de lecture et d’écriture en classe 
lorsqu’ils ont accompli la tache 
qu’on leur a donnée pour le Test de 
rendement. Le contenu et la 
présentation étaient tels que 
Yattention du lecteur était captée 
et maintenue. Tout cela appuie 
lobservation faite par plusieurs 
enseignants que les éléves écrivent 
plus en classe de frangais. La 
plupart des éléves ont de bonnes 
idées qu’ils organisent en un tout 
cohérent. Ils maitrisent assez bien 
les structures de phrases et 
utilisent un vocabulaire approprié 
pour la 6° année. 


L’usage continue d’étre un point 
faible. Dans cette catégorie, moins 
déléves ont atteint le niveau 
d’excellence et une plus grande 
proportion a été jugée limitée. 


Les enseignants qui ont fait la 
correction ont observé que: 


° les éléves savent que ce qu’ils 
écrivent va étre lu et ils écrivent 
avec un public cible en vue et 
avec une intention de 
communication définie. 

¢ 4 mesure que le programme est 

de mieux en mieux implanté, les 

éléves écrivent plus et de facons 
plus variées. 

Pusage continue d’étre un point 

faible. Nos éléves ont des 

difficultés avec certains 
éléments de la langue qui 
devraient étre maitrisés par la 
6° année. 


Partie B: Lecture 

Les correcteurs qui ont aidé a 
confirmer les standards ont trouvé 
que le test de lecture représentait 
une évaluation juste de ’habileté 
de lire pour le niveau de 6° année. 
La variété de genres littéraires, le 
contenu, le niveau de difficulté 
étaient tous appropriés pour les 
éléves de 6° année. Ils ont aussi 
apprécié le montant de littérature 
canadienne dans le test. 


Dans l’ensemble, les éléves ont 
bien réussi le test. Cependant, pas 
assez d’éléves ont rencontré le 
standard acceptable en lecture. 
L’habileté de comprendre ce qu’on 
lit et de pouvoir bien identifier 
Yintention de communication de 
Yauteur ainsi que les idées 
exprimées dans le texte sont 
essentielles. I] faut continuer de 
travailler 4 développer les 
habiletés langagiéres des éléves. 


L’échantillon de questions et 

les commentaires qui suit illustre 
les forces et les faiblesses des 
éléves qui réussissent au standard 
acceptable et au standard 
d’excellence. Pour chaque question, 
la bonne réponse est indiquée par 
un astérisque et le pourcentage 
d’éléves qui ont choisi chacune des 
réponses possibles est donné. I] est 
important de noter que toutes les 
questions étaient basées sur les 
textes présentés dans le test. 
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13. Lacoccinelle utilise ses deux ailes 
cuirassées surtout pour 


15,0 A. seglisser 

69,0 *B. se protéger 

10,0 C. éviter la sécheresse 

5,0 D. vivre sous des tas de feuilles 


48. D’aprés l’auteur, la recette miracle qui 
a changé Timothée était 


62,5 *A. son accueil dans une famille de dix 
enfants 
13,0 B. sa fierté et son indépendance 
16,0 C. son-cours de dressage 
8,0 D._ son intelligence 


22. Quand le poéte dit: “Chacun tracera sa 
voie” (vers 10), il veut dire que chaque 
personne 


10,0 A. veut connaitre son avenir 
68,5 *B. suit son propre chemin 

9,0 C. décide ou elle va vivre 
12,5 D. part ala découverte 


7. — Lauteur dit: “Mais c’est un exemple tres 


différent de réussite que nous a donné... 


Mére Teresa” (lignes 8-11) parce que la 
réussite de Mére Teresa est surtout une 
réussite sur le plan 


31,5 A. intellectuel 
14,0 B. féministe 
3,0 C. matériel 
51,5*D. humain 
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Standard acceptable—exemples de questions et 
commentaires 


La question 13 demandait a l’éléve de juger, selon l'information 
donnée dans le texte a lire, l’utilité principale des ailes cuirassées de la 
coccinelle. La plupart des éléves qui ont réussi au standard acceptable 
ont choisi la bonne réponse. 


La question 48 exigeait que les éléves trouvent, par inférence, une 
des idées principales du texte. Plusieurs éléves qui ont rencontré le 
standard acceptable ont eu de la difficulté 4 choisir la bonne réponse. 


La plupart des éléves qui ont rencontré le standard acceptable avaient 

Phabileté: 

°d’identifier correctement les idées principales et secondaires d’un 
texte de lecture 

de faire des inférences et des jugements de nature peu complexe 


Plusieurs éléves 4 ce niveau avaient de la difficulté a: 

*identifier correctement les idées principales et secondaires quand 
celles-ci étaient moins évidentes 

*faire des inférences et des jugements qui demandaient une réflexion 
plus profonde 


Standard d’excellence—exemples de questions et 
commentaires 


La question 22 demandait aux éléves d’identifier le sens d’une 
expression utilisée dans un poéme. Presque tous les éléves au standard 
d’excellence ont choisi la bonne réponse. 


La question 7 demandait aux éléves de faire une inférence et un 
jugement sur une des idées principales du texte donné. Certains éléves 
au standard d’excellence ont eu de la difficulté & choisir la bonne 
réponse. Trop d’éléves ont choisi la réponse A. 


Conclusion 


Les résultats au Test de rendement présentés dans le rapport 
indiquent que, de facon générale, les éléves des programmes 
francophones réussissent assez bien. La compréhension en lecture 
demeure un domaine a surveiller. I] serait souhaitable de voir plus 
d@éléves atteindre le standard acceptable en lecture 4 l'avenir. 


Appendix A 


Standards. 


The move toward a results-based 
curriculum has re-emphasized the 
need for a clear delineation of 
standards and their purpose. 

All standards and all methods 

of setting standards require 
judgment. 


The process of setting a 
standard can be only as good as 
the judgments that go into it. 
The standard will depend on 
whose judgments are involved 
in the process. In this sense, all 
standards are subjective. Yet, 
once a standard has been set, 
the decisions based on it can be 
made abjectively. Instead of a 
separate set of judgments for 
each test-taker, you will have 
the same set of judgments 
applied to al] test-takers. 
Standards cannot be objectively 
determined, but they can be 
objectively applied.' 


Definitions 

The Achievement Testing Program 
is directly concerned with three 
different but related standards. 
These provincial standards are 
curriculum standards, achievement 
standards, and assessment 
standards. 


¢ Curriculum Standards are 
the expected student learnings 
sequenced into grade levels. 
They include specific statements 
of knowledge, skill, and attitude 
expectations against which 
student performance is judged. 
These standards are established 
in the process of curriculum 


development and are found in 
the Program of Studies 
document produced for each 
course. 


e Achievement Standards are 
judgments that specify what 
percentages of students are 
expected to achieve or exceed an 
acceptable and an excellent level 
of performance in relation to 
each course of studies, i.e., the 
relevant curriculum standards. 
It is important to point out that 
this judgment is not a prediction 
of the percentage of students 
who will actually achieve or 
exceed acceptable or excellent 
levels of performance but rather 
is a specification of the 
percentage of students at a 
given age or grade in school who 
are expected to achieve or exceed 
an acceptable or excellent level. 
These standards apply to school, 
jurisdiction, and provincial 
performance. 


e Assessment Standards are the 
criteria adopted for judging 
actual student achievement 
relative to curriculum standards. 
They are ultimately expressed 
and applied as test scores. They 
are derived from answers to 
questions such as: 


What scores must a student 
obtain or how many 
questions on a given test 
must a student answer 
correctly in order for his/her 


performance on the test to be 
judged as acceptable or 
excellent? 


These standards apply to 
individual student performance. 


Indirectly, the Achievement 
Testing Program influences local 
targets. 


¢ Local Targets are the objectives 
set in schools and jurisdictions to 
assist students in moving toward 
or exceeding the provincial 
assessment standards. These 
local targets reflect the specific 
needs of individuals and groups 
within a specific community. 


The Student Evaluation Branch 

is responsible for establishing 

and reporting the provincial 
assessment standards and shares 
responsibility with the Curriculum 
Branch for establishing provincial 
achievement standards as they 
relate to the Achievement Testing 
Program. 


1 Passing Scores; Samuel A. Livingston, Michael J. Zieky; Educational Testing Service, 1982. 
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Purpose 


The purpose of defining standards, 
and the subsequent process of 
setting the provincial assessment 
standard within the sphere of the 
Achievement Testing Program, is 
to answer questions such as: 


© What is acceptable and 
excellent performance in 
relation to the curricular 
expectations for students at 
the age or grade being tested? 


What percentages of students 

at the age or grade being tested 
ought to achieve or exceed an 
acceptable and excellent level, 
assuming adequate teaching and 
resources? 


What scores on a specific test 
shall reasonably represent 
acceptable and excellent 
performance respectively? 


What are the prevailing 
strengths and weaknesses of 
Alberta students in relation to 
the curriculum being tested? 
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In essence, for each test and level 
of performance, the assessment 
standards setters are challenged 
to answer the question: 


What score must a student 
obtain or how many questions 
must a student answer 
correctly to be judged as 
having achieved an acceptable 
or excellent standard? 


Satisfactory performance may only 
be said to exist when the 
percentage of students scoring at 
or above the assessment standards 
is equal to or greater than the 
achievement standards. 


In this report, tables in sections 2, 
3, 4, 9, and 10 dealing with 
standards show what percentage of 
students achieved or exceeded 
standards on each major 
component and on the total test. 


The Assessment 
Standard-Setting 
Process 


Figure A-1 shows the model 
established in 1991 by the Student 
Evaluation Branch to set 
standards for achievement tests. 
The objective was to widen the 
process of setting assessment 
standards as much as possible 
over previous years and 
especially to provide for 
community input and 
feedback. The process involves 
individuals and groups making 
judgments that contribute to 
establishing the assessment and 
achievement standards. 


Except for the members of the 
Public Advisory Committee, 

the members of all committees 
outlined in Figure A-1 are expected 
to be highly knowledgeable about 
both the curriculum and the 
learning characteristics of the 
students who are writing the tests. 


Figure A-1 
Process Model for Standard Setting 
Achievement Testing Program 

June 1992 


2 


1 
Curriculum and 
Test Developers 
Committee 


Test Review 
Committee 


INPUT 


3 
Public Advisory 
Committee 


6 
Director's 
Final Standards Review 
Committee— 
Representatives 
from 1, 2, 3, 4,5 


PRODUCT 


ASSESSMENT STANDARDS 
Raw scores (cut-off scores) representing 
Acceptable Standard and _~ 


Standard of Excellence 
and 
ACHIEVEMENT STANDARDS 
Percentage of students expected to 
achieve or exceed Assessment Standards 


4 
Teacher Standards 
Committee 


5 
Analytic Services 
Committee 
Student Evaluation 


1.Curriculum and Test 
Developers Committee 
There is one Curriculum and 
Test Developers Committee for 
each achievement test. The 
committee consists of Alberta 
Education consultants, 
curriculum developers, and 
test development specialists. 
Their objective is to recommend 
assessment standards to the 
Final Standards Review 
Committee. They may also 
review and make 
recommendations on 
Achievement Standards. 


2.Test Review Committee 
There is one Test Review 
Committee for each achievement 
test. Experienced and 
knowledgeable educators— 
teachers, superintendents, 
university professors, and 
Alberta Education consultants— 
meet with the developers of 
each test. Their purpose is to 
review the test and make 
recommendations for 
improvements where necessary. 
In addition, the committee 
reviews the appropriateness of 
current course achievement and 
assessment standards. 


3.Public Advisory Committee 
A Public Advisory committee, 
composed of representatives from 
the general public, met again in 
1992 and discussed standards. 
See Appendix B for more 
information on the proceedings 
of this committee. 


4.Teacher Standards 
Committee 
Approximately 20 experienced 
teachers from different areas of 
the province are selected to sit on 
a Teacher Standards Committee 
for each test. To be selected for a 
committee, a teacher must have 
been teaching in the grade and 
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subject area for the previous two 
years. At present, the procedures 
used to aid teachers in setting 
the assessment standards are 
the modified Angoff method for 
Mathematics and Science 
subjects and the Nedelsky 
method for Humanities subjects.” 
The teachers make judgments 
about appropriate assessment 
standards and recommendations 
to the Final Standards Review 
Committee. As well, they review 
Achievement Standards for 
appropriateness. 


5.Analytic Services Committee 
The Analytic Services Unit of the 
Student Evaluation Branch is 
the professional quality control 
and advisory group for the 
complete process of standard 
setting. There is a committee of 
professionals in psychometrics 


and statistics for each test. A 
critical function is to ensure that 
Alberta Education’s standard- 
setting procedures produce 
technically valid results that 
meet the strict requirements of 
high quality professional studies. 
The committee makes 
recommendations for assessment 
standard-setting procedures, 
leads individual group 
discussions, and recommends 
improvements where necessary. 
Another function is to determine, 
independently, assessment 
standards through statistical 
analyses of current student 
achievement data. 


6.The Final Standards Review 


Committee 

The Final Standards Review 
Committee consists of 
representatives from the above 


committees and is chaired by the 
Director of Student Evaluation. 
In separate sessions, the 
recommendations of all test 
committees are presented and, 
through consensus, final 
Assessment Standards are 
adopted for each test. 


1992 Assessment and 
Achievement Standards 


Tables A-1 to A-3 show the 
assessment and achievement 
standards adopted in June 1992 

by the Final Standards Review 
Committee for the grades 3, 6, 

and 9 achievement tests. The 
tables also show the percentages of 
students achieving or exceeding 
provincial assessment standards. 


2 “A Consumer's Guide to Setting Performance Standards on Criterion-Referenced Tests” Ronald A. Berk: Review of Educational 
Research, Spring, 1986, Volume 56. 
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Table A-1 
Grade 3 Social Studies ; 
Assessment and Achievement Standards 
June 1992 
Standard of Excellence 
Provincial Percentage Achieving Provincial 
Assessment* or Exceeding Achievement** 
Category Standard Assessment Standard Standard 
Total Test 47/50 15.9 15 
Concepts 17/18 22.7 15 
Process Skills 30/32 20.9 15 
Acceptable Standard 
Provincial Percentage Achieving Provincial 
Assessment* or Exceeding Achievement** 
Category Standard Assessment Standard Standard 
Total Test 31/50 83.5 85 
Concepts 11/18 81.0 85 
Process Skills 20/32 85.0 85 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the lowest score a 
student must achieve for his/her performance to be judged acceptable or excellent in relation to curricular expectations. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provincial Assessment 
Standard. 
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Table A-2 


Grade 6 English Language Arts 
Assessment and Achievement Standards 


June 1992 
Standard of Excellence 
Provincial Percentage Achieving Provincial 
Assessment or Exceeding Achievement 
Category Standard Assessment Standard Standard 
Total Test 80/100* 9.5 15 
Written Response 28/35 13.6 15 
Reading Skills 40/50 15.3 15 
Acceptable Standard 
Provincial Percentage Achieving Provincial 
Assessment or Exceeding Achievement 
Category Standard Assessment Standard Standard 
Total Test 51/100 75.6 85 
Written Response 18/35 76.1 85 
Reading Skills 25/50 72.7 85 


*The Writing score is multiplied by 50/35 before adding it to the Reading score so that both are weighted equally. 


Table A-3 
Grade 9 Mathematics 
Assessment and Achievement Standards 
June 1992 
Standard of Excellence 
Provincial Percentage Achieving Provincial 
Assessment or Exceeding Achievement 
Category Standard Assessment Standard Standard 
Total Test 42/49 8.9 15 
Problem solving 26/31 10.7 15 
Knowledge 16/18 12.7 15 
Acceptable Standard 
Provincial Percentage Achieving Provincial 
Assessment or Exceeding Achievement 
Category Standard Assessment Standard Standard 
Total Test 23/49 67.4 85 
Problem solving 14/31 64.1 85 
Knowledge 9/18 75.6 85 
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Public Review of 
Standards and Results 


On July 23 and 24, 1992, 
representatives of twenty-four 
business, professional, and 
community organizations reviewed 
achievement test results and 
discussed standards. This meeting 
was planned to ensure that input 
from groups representing “public” 
interests would be considered in 
the overall reporting of results. 
The purpose of the meeting was to 
gather direct public input into the 
question: 


How well should Alberta 
students achieve in Grade 3 
Social Studies, Grade 6 
Language Arts, and Grade 9 
Mathematics? 


The two-day meeting was designed 
to collect information that would 
contribute to a better understanding 
of how well Alberta students are 
doing. This was accomplished by 
exploring and discussing answers 
to four questions. Each question is 
listed below, along with the results 
of the discussions. 


Question: 


What do members of the public 
expect students to know and be 
able to do in Grade 3 Social 
Studies, Grade 6 Language 
Arts, and Grade 9 
Mathematics? 


For each subject area, participants 
listed the knowledge and skills 
that they felt were important 
learnings for students to achieve at 
each grade level. 


The following learnings were 
highlighted by the group as 
important: 


Social Studies 3 
Students should: 


© know the role of business in a 
community 


know the behaviors that 
reinforce working in a group 


know that some communities are 
unique and have an awareness of 
the concept of respect 


be able to use a variety of ways 
to access information, including 
interviewing and basic library 
skills 


¢ be able to read and interpret 
maps and graphs 


be able to apply the understanding 
of the concept of community in 
real-life situations 


English Language Arts 6 
Students should: 


¢ know how to read for meaning 


¢ know how to organize 
appropriate content and use 
coherent sentences to write for a 
purpose 

© be able to use deductive 
reasoning to draw inferences 
from what has been read 


© be able to select and use 
vocabulary that provides detail 
to meaning 


¢ be able to derive meaning from 
context 


Mathematics 9 
Students should: 


¢ know and be able to use the 
steps of the problem-solving 
process 


¢ know and be able to apply the 
formula for calculating area in a 
real-life application 


¢ know how to manipulate simple 
algebraic equations to solve 
problems 


* be able to convert fractions to 
decimals 


¢ be able to use basic statistical 
knowledge such as mean, 
median, mode, and sampling 
techniques to analyze data and 
estimate populations 
respectively 


* be able to calculate discount 
value using their knowledge and 
understanding of ratios and 
proportions 


Question: 


How closely do the public 
expectations about student 
learning relate to provincial 
standards? 


Participants had the opportunity to 
review the achievement tests, the 
student attitudes survey, and the 
performance assessments that had 
been administered in 1992. There 
was essentially a match between 
the participants’ expectations of 
what students should know and be 
able to do, and the knowledge and 
skills students needed to answer 
the questions in each of the tests. 
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Participants expressed concern 
that some of the questions were 
worded in a manner that could 
make questions harder than they 
needed to be. More open-ended 
types of questions were suggested 
to allow students the opportunity 
to explain their answers. 


Question: 


How well did students actually 
perform on the achievement 
tests? 


Participants identified questions in 
the tests that corresponded to 
knowledge and skills they felt were 
most important for students to 
learn. In addition, they discussed 
the percentage of students in 
Alberta that should answer each 
question correctly to meet 
provincial standards. Results 
showing how well students 
actually performed on these 
questions were also presented to 
the group. As well, an initial 
analysis of the composition 
component of the language arts 
test was presented. 


In all subject areas, students 
achieved some of the expectations 
participants set for specific 
learnings. For other learnings, 
achievement was below 
expectation. These areas are listed 
below. 


Social Studies 3 
Met Standards: 


* understand the concept of 
community 


° know the behaviors that 
reinforce group work 


¢ be able to read and interpret 
maps 
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Did Not Meet Standards: 


© be able to use research skills to 
find information 


© understand the concept of 
respect 


Language Arts 6 
Met Standards: 


¢ know how to use words and 
expressions accurately most of 
the time 


¢ be able to make simple 
inferences based on literal 
understandings 


Did Not Meet Standards: 


¢ be able to attend to and identify 
pertinent details of setting and 
scene 


* be able to analyze elements of 
the author’s organization 


® be able to understand cause and 
effect relationships 


¢ be able to attend to and identify 
pertinent details and motivation 
of main and minor characters 


© be able to associate the 
meanings of words, phrases, and 
expressions from context 


* be able to construct meaning, 
make predictions, and draw 
conclusions from synthesizing 
information 


Mathematics 9 
Met Standards: 


° be able to apply knowledge of 
ratio and proportion to solve 
single-step problems in a real- 
life context 


¢ know basic statistical terms and 
be able to apply the knowledge 
to simple problems 


Did Not Meet Standards: 


* know how to convert fractions to 
decimals 


¢ know how to manipulate 
equations 


* be able to apply knowledge of 
statistics to solve multi-step 
problems 


* be able to calculate simple 
interest earned on a principal 
during one year 


* be able to evaluate an 
expression using positive and 
negative integers 


¢ be able to apply the 
understanding of length, width, 
and area to solve multi-step 
problems 


¢ be able to construct a formula 
that shows the relationship 
between two quantities 


Question: 


Did the group feel that 
students’ performance on the 
achievement tests was good 
enough? 


Generally, the participants of the 
two-day meeting felt that student 
performance in Grade 3 Social 
Studies was “good enough” and 
that student performance in Grade 6 
Language Arts and Grade 9 
Mathematics was “not good 
enough.” 


Other Comments from 
Participants 

Several other comments were 
made. All members of the 
committee had an opportunity to 
provide feedback regarding the 
organization and activities of the 
two day meeting. Their comments 
are summarized below: 


¢ All participants indicated that 
they had an adequate 
opportunity to express their 
views. 


¢ Participants also indicated that 
the group discussions were good 
and worked well to achieve a 
consensus. 


¢ Participants felt that 
presentations were well paced 
and timed. 


¢ Many participants indicated 
that the highlight of the two 
days was when they actually 
had to complete the same tasks 
students had to perform. 


* Participants suggested that this 
type of “public meeting” activity 
should continue, and some 
suggested that perhaps it should 
be expanded to the local school 
board level. 


A Final Note 

The views of participants 
regarding the achievement of 
Alberta students were used to help 
set the assessment standards for 
the 1992 achievement tests. 


A more detailed report on the 1992 
meeting was distributed to 
participants and is available upon 
request. Please contact Yvonne 
Johnson at 427-0010 to obtain a 
copy of this report. 


The next meeting for public review 
of achievement standards will be 
held in July 1993. 
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Appendix C 


Guidelines for 
Interpreting Results 


Following each administration 

of the achievement tests, a 
Provincial Report is prepared. 
This report is a public document 
that describes the aggregated 
results obtained by those students 
who wrote achievement tests in a 
given year. School board members, 
superintendents, principals, and 
teachers can use the Provincial 
Report as they review results for 
students in their own jurisdictions 
and schools. By using the 
Provincial Report in this way, 
policymakers and educators can 
check their perceptions of local 
achievement against province-wide 
standards and trends in the levels 
of achievement. 


This Provincial Report describes 
the results achieved by students 
who wrote the June 1992 
achievement tests in Grade 3 
Social Studies, Grade 6 English 
Language Arts, Grade 6 French 
Language Arts, and Grade 9 
Mathematics. 


Individual jurisdiction and 
school reports are sent to 
superintendents and principals 
approximately three months before 
the Provincial Report is published. 
Policymakers and educators in 
each jurisdiction are encouraged to 
study carefully the provincial 
results and their own test results. 


It should be noted that results for 
groups of fewer than 25 students 
should be interpreted with caution. 
Results for such groups can be 
affected by the presence of a few 


extreme scores when the analyses 
are performed to produce the 
respective group statistics. 
Consequently, any generalizations 
may appear to indicate performance 
that is higher or lower, better or 
worse than it is in relation to the 
criteria applied. 


Educators at the school and 
jurisdiction levels can make three 
kinds of comparisons to decide if 
the achievement of their students 
is “good enough.” One comparison 
is in relation to expectations or 
standards; another is in relation to 
the achievement of students in the 
entire province of Alberta; and the 
third is to look at the jurisdiction 
results in 1988, when tests in these 
subjects were last administered. 
After making these comparisons, 
teachers, principals, and 
superintendents can identify the 
strengths in the programs that 
were delivered in their grades 3, 6, 
and 9 classrooms. Where the 
results show weaknesses, changes 
can be made where possible and 
desirable. 


Use of the Reports 


The reports are not intended to be 

used as the basis for 

° making decisions about student 
placement or promotion 

* evaluating teacher performance 

* comparing performance between 
or among schools. 


Administrators in each jurisdiction 
should apply separate locally 
developed teacher, school, and. 


school system evaluation policies to 
the tasks of evaluating teacher and 
school performance. 


The information provided in the 
reports is factual regarding what 
the test results are. The 
interpretation of this information 
—hypothesizing why results are as 
they are— involves consideration 
of the many factors and variables 
that contribute to achievement. 


Moreover, the information in these 
reports is limited to selected 
objectives of the Program of 
Studies. Many important aspects 
of learning cannot be measured by 
the time-limited, paper and pencil 
achievement tests. 


Assessment Standards 
and Achievement 
Standards 


Standards have been confirmed for 
each achievement test and are 
reported both here in the 
Provincial Report (sections 1, 2, 3, 
4, 9, 10, and Appendix A), and in 
table 2 of the jurisdiction and 
school reports. Criteria 
representing a standard of 
excellence and an acceptable 
standard were confirmed for each 
of the major reporting categories 
(components) of a test and for the 
total test. These criteria are 
referred to as assessment 
standards. The assessment 
standards are supplemented by 
achievement standards, which 
specify the percentage of students 
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in the province expected to meet or 
exceed the assessment standards. 
Appendix A provides more detail 
on both the standards and the 
process by which they have been 
confirmed. 


Tables in the subject sections of 
the Provincial Report show the 
number of students writing the 
test who met or exceeded the 
provincial assessment standards, 
the number who were expected to 
meet or exceed the standards, and 
the percentages who met or 
exceeded the standards on each 
major component of a test and on 
the total test. 


The school and jurisdiction tables 
also show whether the number of 
students in that school or 
jurisdiction who achieved or 
exceeded the standards is 
significantly different from the 
expected number. A 95% 
confidence interval is used for these 
purposes; this criterion means that 
differences are only reported when 
there is a 5% or smaller probability 
that a difference of that size could 
occur by chance. For schools and 
jurisdictions with fewer than six 
students, significances are not 
calculated and the indications are 
omitted. Although the statistical 
tests take the number of students 
into consideration, it is a useful 
rule of thumb that results for 
‘groups of fewer than 25 students 
must be interpreted with 
particular caution. Chance 
variation in small groups is 
greater. 


Educators interpreting these 
reports are encouraged to consider 
how well their students have done 
compared to the assessment 
standards and how well schools 
and jurisdictions have done 
compared to the achievement 
standards. 
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Comparing Results to 
Average Scores 


While overall test results are 
presented in relation to provincial 
standards, each jurisdiction and 
school report also provides 
jurisdiction or school average 
scores for each reporting category 
or subtest. Educators may 
compare each of these scores to the 
provincial average for the same 
reporting category or subtest to 
determine if differences exist. 


Evaluating the importance of 
differences that may exist between 
jurisdiction or school averages and 
provincial averages requires 
careful consideration of the 
practical significance of differences 
and whether the students should 
be expected to have the same level 
of achievement as other students 
in Alberta. Consequently, we 
suggest that more attention be 
given to comparing achievement 
with standards. 


Factors Limiting the 
Interpretation of Test 
Results 


Educators who are interpreting 
results must take into account the 
following limitations: 


1. Paper and pencil tests 
necessarily measure reading 
achievement in the content area 
being tested. Standards built 
into each achievement test 
reflect the reading level 
expectation for the grade level 
tested. Jurisdictions should 
consider the average reading 
level of their grades 3, 6, and 9 
students, as reading levels below 
these grades will have an effect 
on test results. 


2.If more than 10% of eligible 
students in a jurisdiction did not 


write a test, the reported 
statistics for that jurisdiction 
may not accurately represent the 
true level of achievement. 


3.Consideration should be given to 
the degree to which students in 
particular classes or grades were 
motivated to perform to their 
level of ability. 


Factors That May Affect 
Student Achievement 


Results on the achievement tests 
may have been affected by: 


1. Environment 
*community environment 
eschool environment including 
staff morale 
*socioeconomic background 
efamily circumstances 


2. Personal Factors 
*ability 
*attitude 
emotivation 
®aspiration 
*academic background 
elearning style 


3. Availability and Appropriateness 
of Resources 
¢ programs of study 
*curriculum guides 
*resource materials 
*library services 
ecurrent textbooks 
*references 


4. Instruction 

*teacher qualifications (i.e., to 
teach a particular subject) 

*teacher experience 

*teacher morale 

¢professional development 

estaff turnover 

° professional support 

*teaching strategies 

*timetabling constraints or 
influences 


A Systematic Approach 
to the Effective Use of 
Test Results 

Achievement test results can be 


used constructively as one means 
of improving the quality of 


education. A systematic use of the 


test results would include the 
following steps: 


1.Comparing test results for a 
jurisdiction or school to the 
provincial results. Be sure that 
your comparisons include the 
* total test score, 

* total and subtest scores for 
multiple-choice questions, 

* total and subtest scores for 
written-response assignments 
(when appropriate), 

* individual multiple-choice 
question results, and 

* individual written- 
response question results 
(when appropriate). 


N 


and/or interrelationships in the 
results. 


3. Hypothesizing relationships 
between your observations and 


any of the factors listed in these 
guidelines that may have had an 


effect on achievement or 
achievement test results. 


4, Considering and implementing a 


plan that will help to improve 
the quality of education for 
students. 


An Administrative Model 


for the Effective Use of 


Achievement Test Results 
The following model may be useful 


for those who wish to develop a 
constructive system for 
interpreting achievement test 
results. 


. Noting any patterns, anomalies, 


Basic Principles 

1.It is desirable and feasible for 
teachers and school 
administrators to make use of 
achievement test results in 
analysing the performance of 
their own students. 


2. It is more constructive for 
schools to develop their own 
analyses, interpretations, and 
action plans than to have these 
imposed externally. 


3.The impact of factors such as 
those listed in these guidelines 
should be analysed and 
discussed when reviewing 
achievement test results. 


4. Subtest or reporting category 
results are usually more 
informative than total test 
scores. 


5. Generalizations should be stated 
with caution and should be 
supported by evidence that is 
independent of achievement test 
results. 


6. It is neither desirable nor 
productive to compare the 
results of different schools. 


7. Achievement tests measure 
many of the objectives specified 
by the curriculum. However, 
skills and concepts that are not 
measured by the achievement 
tests are also to be taught and 
evaluated at the local level. 


8. Staff discussions as well as 
written reports are useful means 
of ensuring that results are 
appropriately interpreted and 
used. 


Suggested Content for 
Interpretation of Individual 
School Results 


1. 


10. 


11. 


Subject, grade level, and 
date of achievement test 
administration 


. Number of students who wrote 


the achievement test 


. Profiles of students or groups 


who wrote the achievement 
test, which include 
—previous performances 
—number of students 
repeating the grade 


. Program emphases, such as 


hours of instruction, skill and 
content emphases 


. Instructional practices, such as 


methodology, resources, and 
the relationship between the 
program offered and the 
provincial curriculum 


. Program objectives not 


measured by the achievement 
test 


. School results compared with 


provincial results on subtests 


. Current school results 


compared with those of 
previous administrations 


. Discussion of item results, 


identification of common 
student errors, and 
suggestions of ways for 
reducing the misunderstanding 
that leads to these errors 


Recommendations for the 
following year or semester 


Summary and concluding 
comments 
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Suggested Reporting Structure 
1. Teachers and/or the principal 
analyze the results and prepare 
a written report about each 
administration of an 
achievement test. 


2. The principal reviews and signs 
the report. 


3. The report is shared with 
central office supervisory 
personnel. 


4. The appropriate central office 
supervisory personnel prepare a 
written response to the report, 
with copies of the response 
going to the teachers and the 
principal. 


5. If possible, all staff involved 
meet to discuss the report and 
the response. 


6. Reports are used to improve the 
program and maximize future 
opportunities for student 
success. 


7. When large differences exist 
between expected and actual 
achievement test results over 
time, consideration should be 
given to conducting a formal 
program evaluation. 
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Appendix D 
| Reporting to Parents: 


Answers to Frequently. 
Asked Questions 


What are the achievement tests? 
The achievement tests are 
provincial government tests 
administered in Alberta schools to 
students in language arts, social 
studies, science, and mathematics. 


What is the purpose of the 
achievement tests? 

The achievement tests help 
Alberta Education to communicate 
provincial expectations and results 
for levels of student performance in 
language arts, social studies, 
science, and mathematics. The 
tests enable Alberta Education to 
monitor the level of achievement of 
students throughout Alberta. 

The results also help local school 
boards, principals, and teachers 
identify the strengths and 
weaknesses in their implementation 
of these programs. 


How many achievement tests will 
my child have to write? 

In 1992, students wrote only one 
test in Grade 3 social studies, one 
in Grade 6 language arts, and one 
in Grade 9 mathematics. In 1993, 
the program will assess Grade 3 
students in language learning, 
Grade 6 students in social studies, 
and Grade 9 students in science. 
In addition, the Grade 3 Language 
Learning Achievement test will be 
given to a provincial sample of 9 
year-olds not registered in Grade 3 
during the 1992-93 school year. 


How should I prepare my child to 
write an achievement test? 

No preparation beyond normal 
classroom instruction is required to 
write an achievement test. While 
students should be encouraged to 
do their best, a good night's sleep 
and a relaxed, confident approach 
to testing are the best possible 
preparation. 


How much do these tests count 

for my child? 

The achievement tests do not 
affect students’ final marks. The 
classroom teacher is responsible for 
evaluating students and awarding 
final marks. Achievement test 
results are not released by Alberta 
Education until October, long after 
students’ marks have been 
determined by the classroom 
teacher. 


How do achievement test results 
help classroom teachers? 
Achievement test results provide 
feedback on student achievement 
to school boards, principals, and 
teachers. For example, teachers in 
a school where student performance 
is high in one skill area but low in 
another may wish to examine their 
programs to see if changes are 
needed to achieve a better 
instructional balance. 


What are the limitations of the 
achievement tests? 

Paper and pencil tests cannot 
easily measure such things as 
laboratory skills, small group 
discussions, and creative thinking. 
Thus, some student strengths can 
be evaluated only by the classroom 
teacher. Also, a single test cannot 
reveal as much about a student's 
development and growth as can 
evaluation by the classroom 
teacher over the course of a full 
school year. 


What advantage do achievement 
tests have over other standardized 
tests? 

Unlike commercially developed 
tests, achievement tests are based 
on learning expectations and 
standards communicated through 
Alberta’s programs of study. The 
tests are designed, written, and 
evaluated by experienced 
classroom teachers from across 
the province. Tests developed 
elsewhere may not reflect 
curriculum or standards 
appropriate for Alberta. 


How do I interpret achievement test 
results? 

The Achievement Testing Program 
Provincial Report includes 
guidelines for interpreting results. 
Readers are cautioned not to 
overgeneralize conclusions based 
on a single administration of the 
test. Results should be compared 
with expectations or with the 
results of previous achievement 
tests in the same subject. Any 
trends that are observed in the 
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scores must then be interpreted in 
the context of a variety of factors 
that could affect student 
achievement. 


Comparisons between districts, 
schools, or classrooms are likely to 
prove misleading and are therefore 
discouraged. 


Can I find out how my child did on 
the achievement test? 

Yes, by contacting the school 
where your child wrote the test. 
Individual results on the 
achievement tests are made 
available to school principals in the 
fall. Since the tests are designed 
to gather information on groups of 
students, not on individuals, 
individual results must be 
interpreted with caution. 


Where can I get additional 
information about the Achievement 
Testing Program? 

Bulletins describing the content of 
the coming year’s achievement 
tests and the Provincial Report 
describing the results of the 
previous year’s testing are 
distributed to schools each year. 
Requests for copies of these 
publications or questions and 
comments regarding the 
Achievement Testing Program 
should be directed to: 


Mr. Dennis Belyk 

Assistant Director 

Achievement Testing and 
Diagnostic Evaluation Programs 
Student Evaluation Branch 
Alberta Education 

Devonian Building, West Tower 
11160 Jasper Avenue 
Edmonton, Alberta T5K 0L2 
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Appendix E 


Developing 
Achievement Tests 


The Student Evaluation Branch 
develops achievement tests that 
measure student achievement at 
the grades 3, 6, and 9 levels. 
Province-wide testing in language 
arts, mathematics, science, and 
social studies follows a four-year 
cycle for each grade level and 
subject. Many individuals and 
groups are involved in the 
development of each test: 
practising classroom teachers, 
school and central office 
administrators, and representatives 
of post-secondary institutions, the 
Curriculum Branch, the Language 
Services Branch, Regional Offices, 
and the Student Evaluation 
Branch. Student Evaluation 
Branch staff ensure the 
development of valid and reliable 
tests. 


The following is a summary of the 
phases of the test development 
process: 


¢ Planning 
¢ Approving Blueprints 
* Developing Test Questions 


¢ Constructing and Administering 
Field Tests 


¢ Analyzing and Revising 
¢ Constructing Final Field Tests 
¢ Approving Final Field Tests 


¢ Administering Final Field 
Tests 


* Constructing the Final Test 


¢ Preparing and Administering the 
Final Test 


° Marking 


¢ Analyzing and Reporting the 
Results 


Under normal circumstances, it 
takes three years to complete all 
phases of the process. 


Planning 


Test developers ensure that the 
design of each achievement test 
reflects the learning expectations 
in the Program of Studies for each 
subject. Planning takes into 
consideration those parts of the 
program that are testable in a 
paper and pencil format, within a 
given time frame. Teachers and 
consultants from across the 
province assist in preparing the 
design of each test. 


Test developers prepare an interim 
test blueprint (an overall plan used 
to guide the development of a test). 
Questions that must be addressed 
at this point are: 


¢ What knowledge, skills, and 
attitudes should students be 
expected to demonstrate? 


¢ What types of questions will 
constitute the test (machine- 
scored, short answer, or 
extended written response)? 


What weighting will each part 
of the test be given? 


How long and how demanding 
should the test be? 


* How should the results of the 
test be organized for reporting 
purposes? 


In order to ensure that each test 
will produce meaningful and 
reliable results, test developers 
incorporate statistical as well as 
curricular standards in the test 
design. Statistical standards 
address areas such as range of 
question difficulty and the 
requirements for reporting. 


Each dimension of the curriculum 
for which results are reported must 
contain at least six questions if the 
results are to be meaningful. 


Approving Blueprints 


Blueprint approval establishes the 
overall design of the test, the exact 
emphases given to each category 
for which results are reported, and 
the emphases given to the different 
cognitive levels. 


The interim blueprint is reviewed 
by a committee of Alberta 
Education personnel that 
represents the Curriculum Branch 
(or Language Services Branch), 
Regional Office consultants, and 
the Student Evaluation Branch. 
This committee makes 
recommendations to the Director 
of the Student Evaluation Branch. 
The blueprint recommended by 
the Alberta Education committee 
is then reviewed by a Test Review 
Committee, which consists of 
teachers and members nominated 


123 


by the Conference of Alberta 
School Superintendents and post- 
secondary institutions. This 
committee makes recommendations 
to the Director of the Student 
Evaluation Branch. 


Developing Test 
Questions 


Following blueprint approval, 
committees of practising classroom 
teachers working at the 
appropriate grade level are 
formed, and question development 
meetings are held. These 
committees develop new test 
questions that reflect the learning 
expectations of the Program of 
Studies and curriculum 
specifications. Where necessary, 
question developers are trained 

in the principles of question 
construction. Questions built in 
committee are then screened for 
format, validity, blueprint ‘fit’, and 
other design considerations. 


Constructing and 
Administering Field 
Tests 


After careful editing and 
formatting of questions developed 
by the teacher committees, field 
tests are constructed. Any 
required artwork is completed 
during this phase of the test 
development process. 


With permission from school and 
jurisdiction personnel, field tests 
are sent to a number of teachers 
throughout Alberta. The students 
involved are representative of the 
student population for which the 
test has been designed. A 
minimum sample of 150 students 
writes each field test. 
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Teachers who administer a field 
test are asked to comment in 
writing on the following: 


* reading level 


* how closely the question matches 
the way in which a concept was 
taught 


* level of difficulty of the questions 


* quality of the questions and 
graphics 


° errors of any kind 


The results from the administration 
of this initial round of field tests 
are used to validate content, to 
determine difficulty levels, and to 
ensure that questions are 
expressed clearly. Special field 
tests are also constructed to ‘try 
out’ new assessment strategies and 
techniques that may be useful for 
future assessments. 


Analyzing and Revising 
The results of each field test are 
then analyzed and scrutinized to 
determine whether individual 
questions require revision. 
Teacher comments regarding the 
way that test questions are 
structured and the way that a 
subject is being taught are also 
carefully considered and used to 
guide revision. 


Questions deemed to require 
changes are revised and submitted 
for further field testing. 


Constructing Final 
Field Tests 


Once the initial field test results 
are thoroughly analyzed and 
questions requiring changes are 
revised, final field tests are 
constructed. These field tests 
follow the approved blueprint and 
parallel the actual achievement 
test in format and design. 

Final field tests, like all field 
tests, are submitted for further 
validity checking, editing, and 
proofreading. In grades 6 and 9, 
separate tests in English and in 
French are developed for language 
arts. At this point, all other tests 
for Grade 6 and Grade 9 are 
translated into French. 


Approving Final 
Field Tests 


After the final field tests have been 
constructed, a second meeting of 
the Alberta Education Committee 
that represents the Curriculum 
Branch (or Language Services 
Branch), Regional Office 
consultants, and the Student 
Evaluation Branch is convened. 
This committee reviews the 

final field tests and makes 
recommendations for improvement. 
The Test Review Committee, which 
approved the blueprint in Phase 
Two of the test development 
process, meets a second time to 
review and recommend for 
approval the final field tests and 
the instructions for administering 
the tests. If a test includes short- 
answer or extended-writing 
questions, the Test Review 
Committee discusses standards 

of achievement and marking 
standards appropriate for the test. 
Again, this committee makes 
recommendations to the Director 
of the Student Evaluation Branch. 


Administering Final 
Field Tests 


The final field tests are 
administered and the results are 
used as a final screen in selecting 
questions for placement on the 
provincial achievement test. A 
minimum sample of 250 students 
writes each final field test. The 
sample is selected to include: 


* only students who have received 
instruction in the course 


¢ students representing a normal 
distribution of ability levels 


students from rural and urban 
schools 


¢ students from large and small 
schools 


Constructing the 
Final Test 


The construction of the final test 
form is based upon information 
collected from the final field test 
administration. The Test Review 
Committee is reconvened to review 
the final test form and to assist in 
setting assessment and 
achievement standards. 


The test is submitted for final 
validity checking, editing, and 
proofreading. Grade 6 and Grade 9 
achievement tests, in subjects 
other than language arts, are 
translated into French. 


For each test, an information 
bulletin is prepared that outlines 
the design and nature of the 
upcoming tests. These bulletins 
are distributed to each school at 
the beginning of the school year 

to facilitate program and 
instructional planning by teachers 
and administrators. 


Preparing and 
Administering the 
Final Test 


The completed achievement test is 
commercially printed and prepared 
for distribution. Sufficient copies 
of the test are mailed to each 
school. Quantities are based on 
the number of students enrolled in 
the subject as reported to the 
Student Evaluation Branch. The 
test is administered to students by 
their classroom teachers. 


Marking 


All written-response sections of 
the tests are marked by classroom 
teachers. These teachers, who 
are recommended by their 
superintendents, are currently 
teaching the course being 
evaluated, have taught the course 
for a minimum of two years, and 
hold a valid Alberta Permanent 
Professional Certificate. 


Student Evaluation Branch staff 
train and supervise the teachers 
during the marking sessions. 

All multiple-choice and numerical 
responses are machine scored. 


Analyzing and Reporting 
the Results 


A results report is prepared and 
distributed to superintendents, 
school principals, Alberta 
Education officials, and other 
Departments of Education. This 
report is also made available to the 
general public. In addition to the 
Achievement Testing Program 
Provincial Report, each school and 
jurisdiction receives a statistical 
summary for its student 
population. 


For further information, please 
refer to the Achievement Test 
Bulletins or call the Assistant 
Director, Achievement Testing and 
Diagnostic Evaluation Programs, 
at 427-0010. 
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Achievement Testing Program The Student Evaluation Branch 
Provincial Report strives to produce documents 


that are useful to educators and 
their communities. The purpose 


and format of the next report are 
reviewed. Please take a moment to 
respond to the questions and send 
to: Assistant Director, Analytic 


of the following questionnaire is Services 
to collect your opinions about the Student Evaluation Branch 
Achievement Testing Program Alberta Education 
é ‘ Provincial Report. All opinions 11160 Jasper Avenue 
Que S t tonnatre are considered when the content Edmonton, Alberta T5K 0L2 
Content of the Report 


1. Please judge the usefulness of the information included in the various sections of the report by checking 
the appropriate boxes below. 


Very Adequately Partially Not 


Useful Useful Useful Useful 
Section 1: Summary of Achievement Test Results 
Section 2: Grade 3 Social Studies 
Section 3: Grade 6 English Language Arts 
Section 4: Grade 9 Mathematics 
Section 5: Achievement by Gender bse L__| — 
Section 6: Achievement by Age LJ L353 L__| aoe 
Section 7; Achievement by Grade Level E21 L__} — | 
Section 8: Achievement Over Time ee L__] | 
Section 9: Grade 6 French Language Arts—French 
Immersion 
Section 10: Francais 6¢ Année—Programme 
francophone 
Format of the Report 


1. Please judge the usefulness of the report’s format by checking the appropriate boxes below. 


Organization into Separate 


Sections 


Three-Column Presentation 


of Text 


Presentation of Figures 
Presentation of Tables 


Blending of Information 
in Text, Figures, and Tables 


Very Adequately Partially Not 


Useful 


Useful Useful Useful 


Please write on the back of this page if you wish to comment further on the report. 
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Pik oe dine tease wee oe 


2h Sgt ee 


